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Preface 



In recent years, health educators have increasingly recognized that systematic evaluation 
can help them appraise and improve their programs* Foi this potential to be realized, 
however, effective mechanisms for gathering relevant data are required. In the past, critical 
information about a program's effects was not collected in some instances because suitable 
measures for gauging those effects were lacking. The purpose of this handbook is to rectify, 
at least in part, this deficiency in the evaluation of health education programs dealing with 
stress management. 

This book is one of seven health education evaluation handbooks resulting from a project 
jointly initiated in 1980 by the United States Centers for Disease Control (CDC) and the 
Office of Disease Prevention and Health Promotion (ODPHP) of the Office of the Assistant 
Secretary for Health. The handbook is not intended to be prescriptive or all-inclusive. Those 
who evaluate stress management programs should regard the handbook as only a resource, 
that is, a collection of assessment tools that maybe of use in program evaluation. The extent 
to which the handbook will actually be useful depends chiefly on the extent to which it 
contains assessment tools that correspond to the evaluation needs of a particular stress 
management program. 

Handbook Development 

This handbook has been created by lOX Assessment Associates (lOX), selected 
competitively on the basis of responses to a governmentally issued request for proposals. 
lOX was to collect and develop program evaluation measures for critical behavioral, 
knowledge, skill, and affective outcomes in the area of stress management. Three panels of 
experts played prominent roles in the creation of this handbook. A Handbook-Development 
Panel, consisting of six experts familiar with stress management programs or their 
evaluation, guided the initial development of the handbook. The Handbook-Development 
Panel identified important outcomes for stress management programs. lOX staff, drawing 
on the advice of paneHsts, then developed assej^sment instraments to assess panel-identified 
program outcomes. The names and affiUations of the Stress Management 
Handbook-Development Panelists are provided below: 



Handbook-Development Panel 



Dr. Barbara Estabrook 
Multi-Group Health Plan 
Wellesley, Massachusetts 



Dr. Donald Iverson 
University of Colorado 
Denver, Colorado 



Dr. E.M. Gherman 
American Institute of Stress 
Yonkers, New York 



Dr. Jason Millman 
Cornell University 
Ithaca, New York 



Dr. Daniel Girdano 
ECOTOPIA Institute 
Winter Park, Colorado 



Dr. Paul Rosch 

American Institute of Stress 

Yonkers, New York 



The Handbook-Development Panel met at the beginning of the project in order to 
isolate the chief outcomes that stress management programs could reasonably be expected 
to promote. Preliminary statements reflecting these outcomes were identified by the 
panelists. These preliminary outcome statements were refined by lOX staff and mailed to 
the panelists and other interested specialists, all of whom rated the importance of each 
statement. The list of high-priority outcomes that resulted was used to guide the selection 
and development of the original handbook's measures. 

All newly developed measures were mailed to the panelists for review. In addition, all of 
these measures were tried out with small groups of respondents. The measures were revised 
based on the informal tryouts and the panelists' review comments. All of the new measures 
were also reviewed by lOX staff in an effort to eliminate any potential ethnic, gender, 
religious, or socioeconomic bias. 

A completed version of the stress management handbook was delivered to the 
government in 1983. Several thousand copies of the handbook were released by CDC and 
ODPHP to health educators throughout the nation. 

Handbook Revision 

Subsequent to the initial distribution of the handbook, CDC issued, in concert with (1) 
ODPHP and (2) the National Institutes of Mental Health (NIMH), a second request for 
proposals which led to the comprehensive revision of the existing stress management 
handbook. To guide the review and revision of the stress management handbook, a 
Handbook-Revision Panel was constituted. Members of the panel were selected because of 
their dual expertise in (1) the field of stress management and (2) measurement of the 
outcomes sought by stress management programs. Members of the Handbook-Revision 
Panel and their affiliations are listed below: 

Handbcok-Revision Panel 

Dr. Jonathan E. Fielding 
U.S. Corporate Health Management 
and University of California 
Los Angeles, California 

Dr. Susan Folkman 
University of California 
Berkeley, California 

Dr. Leo Goldberger 
New York University 
New York, New York 

The Handbook-Revision Panel met on two occasions. In these meetings panelists 
reviewed the contents of the initial version of the stress management handbook, particularly 
its measures, and suggested deletions, modifications, or additions. Panelists also provided 
giidance regarding ways of making the handbooks more usable to practitioners. During 



Dr. Edward Katkin 

State University of New York 

Buffalo, New York 

Dr. Richard S. Lazarus 
University of California 
Berkeley, California 

Dr. Rudolph H. Moos 
Stanford University 
Stanford,California 
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both of these meetings, the panelists were attentive to the accuracy of the handbook's 
contents. Considerable content, in the measures as well as the introductory' materials, was 
revised or deleted on the basis of panelists' suggestions. 

Overall Guidance 

A third panel, the Project Advisory Panel, provided overall guidance to lOX staff during 
the final three years of the project. These individuals offered technical counsel and strategic 
advice during the revision of all handbooks. Members and affiliations of the Project 
Advisory Panel are listed below: 
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CHAPTER ONE 

A Resource for the Evaluation 
of Stress Management Programs 
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A Resource for tiie Evaluation 
of Stress Management Programs 



This handbook is intended to help those individuals who wish to evaluate health 
education programs dealing with stress management. More specifically, the handbook 
provides a series of measuring devices that, if selected and used judiciously, can improve the 
quality of such evaluations. As a consequence, not only will the technical quality of the 
program evaluation be improved, but any program-related decisions based on the 
evaluation's results are apt to be more defensible. 

An Evidence-Oriented Era 

In recent years, educators have experienced substantially increased pressures to produce 
evidence that their programs are functioning effectively. In contrast to an earlier era when 
it was widely thought that most educational programs were worth the money they cost, 
today's educators find that they are constantly called on to justify the effectiveness of their 
programs. 

The kinds of evidence that health educators have been required to assemble regarding 
program effectiveness have, almost without exception, involved the use of various Idnds of 
assessment instruments. Consonant with that requirement, this handbook contains 
numerous tests and inventories designed to secure the evidence needed to judge the 
effectiveness of stress management programs. The handbool-'s measuring instruments were 
created specifically to assess important goals of the most common types of stress 
management programs offered for adults (in industrial or clinical settings) and for children 
(in school- related programs). 

The handbook, accordingly, makes available to those who operate stress management 
programs the assessment tools by which the effectiveness of such programs can be 
determined. The evidence of program effectiveness currently being demanded of stress 
management personnel can, therefore, be provided by appropriate use of the handbook's 
assessment instruments. Moreover, as will be indicated shortly, appropriate use of the 
handbook's numerous assessment devices can substantially improve the design of stress 
management programs. 

Measurement and Program Design 

Historically, assessment devices have been thought of as instruments to be used after a 
program was concluded. Teachers, for example, have traditionally administered tests after 
instruction was over in order to grade students. Tjwever, even though assessment 
instruments have often been post-instruction creations of instructors, such instruments can 
make important -often overlooked - contributions to the original design of an instructional 
program. Properly developed assessment tools, in fact, can contribute to program design in 
two significant ways. 

First, because assessment instruments are typically intended to measure outcomes of 
interest, such assessment instruments provide program personnel with a ^unge of potential 
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outcomes. An increased range of possible program outcomes generally leads to the selection 
of more defensible outcomes for health education programs. To illustrate, there may be an 
assessment instrument dealing with an attitudinal dimension that, were it not for the 
measuring instrument's availability, might have been overlooked by the program staff. 
Stimulated by the assessment tool's availability, however, the program otaff can add the 
attitudinal dimension to the program's targeted outcomes. 

A second program-design dividend of properly constructed assessment tools is that they 
clarify intended program outcomes and, there'^y, make possible the provision of more 
on-target program activities than would have been the case had such clarification not been 
present. To illustrate, suppose that program personnel intend to feature in their evaluation 
an assessment device focused on a specific stress-management skill. By becoming familiar 
with the composition of that assessment tool, the program staff can be sure to incorporate 
truly relevant practice sequences in their instructional program. Provision of appropriate 
instructional practice for participants need not reflect "teaching to the tesf in the negative 
sense that instructors coach students for specific test items. Instead, providing relevant 
practice so that program participants attain the program's intended outcomes constitutes an 
efficient and effective, research-supported form of instruction. 

To review, then, the measuring instruments provided in this handbook are intended to 
assist those who design and these who evaluate stress management programs. With respect 
to program evaluation, the measures will yield evidence by which to improve programs as 
well as determine program effectiveness. With respect to program design, the measures 
provide a menu of potential program options and, once having been selected, enhanced 
clarity regarding the nature of the outcome(s) sought. 

What the Handbook Contains 

There are several key ingredients in this handbook. It should, therefore, prove helpful to 
readers if the handbook's major sections are presented. Briefly, then, here is a description 
of the handbook's major components: 

Introductory information. In Chapter One, an introduction to the handbook is provided. 
Because the handbook is intended to be used with stress management programs, the chapter 
concludes with a series of evaluation-related issues specific to health education programs 
dealing with stress management. 

Program evaluation essentials. Although a number of people who use this handbook will 
already be familiar with the nature of program evaluation, many handbook-users will not be 
well versed in the conduct of program evaluations. Accordingly, in Chapter Two, an 
introduction is provided to the key operations involved in program evaluation. Although 
space limitations preclude a detailed exposition of all aspects of program evaluation, 
emphasis is given to the role that assessment instruments play in the gathering of 
information needed for defensible evaluations. 

Assessment instruments. Chapter Three contains the handbook's most important 
components, namely, the measuring tools designed to be used in the evaluation and design 
of stress management programs. These measures deal with behavior, knowledge, skill, and 
affective outcomes. The Behavior measures focus on actual behaviors of program 
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participants. Knowledge measures are concerned with participant mastery of a defined set of 
information. Skill measures deal with cognitive, that is, intellectual competencies to be 
mastered by program participants. Finally, affective measures assess participants* altitudes 
and values. 

Each measure is introduced by a brief description of the purpose of the assessment 
instrument, as well as procedures for administering, scoring, and analyzing the resulting 
data. All measures have been provided on separate detachable pages. At the beginning of 
Chapter Three, an overview description of the chapter's measures is provided to facilitate 
the selection of measures. 

Loccd measure appraisal Although the measures contained in this handbook have been 
created with considerable care and were pilot tested in small-scale tryouts, the measures 
have not yet been subjected to a formal empirical appraisal of their technical adequacy. 
Thus, in Chapter Four, a description is provided of how such technical appraisals of t«.- 
handbook's measures can be carried out. 

Annotated bibliography. Because evaluators and designers of programs in stress 
management may wish to consult additional sources regarding program design and 
evaluation, an annotated bibliography is provided to facilitate the handbook-user's selection 
of such materials. (See Appendix C.) 

Amplified content descriptors. The information eligible for inclusion in the knovJcJge 
measures is provided in the handbook's appendix as amplified content descriptors. (See 
Appendix A.) Additional content, that can be used for the generation of new items, is also 
presented. However, these descriptors are r^ot exhaustive accounts of stress management 
content. 

How to Use the Handbook 

The particular ways in which the handbook is used will vary from setting to setting and 
from user to user. For instance, if a handbook-user is relatively unfamiliar with the core 
notions in program evaluation, then a thorough reading of Chapter Two s treatment of 
program evaluation essentials is warranted. In addition, farther reading based on the 
evaluation-related references included in the annotated bibliography would also seem 
useful. 

For handbook-users more familiar with program evaluation, primary attention will 
probably be focused on Chapter Three's measures. Although use of the measures will vary 
from situation to situation, a common four-step usage pattern is depicted in Figure 1.1. 

Note, that in Step 1, the measures are used to represent a range of potential program 
objectives. Clearly, an expanded range of options can lead to more appropriate decisions 
regarding what program objectives to pursue. In Step 2, after the measures for possible 
program evaluation have been reviewed, one or more measures are selected for use in the 
evaluation of the program. In Step 3, after the program evaluation measures have been 
selected, the program staff studies the measures intensively to discern if there are program 
design implications to be drawn from the measures. In Step 4, the measures are 
administered using one of the evaluative data-gathering designs described in Chapte* Two 
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results* 




> 


> 



Figure LI: A four-step usage pattern of the handbook's measures 

and scored according to the scoring directions in Chapter Three, Finally, interpretations of 
the results are made. 

It is important to remember that the handbook's measures are to be used for program 
evaluation, not individual decision making* Thus, if one of the handbook's affective 
measures was used on a pretest-posttest basis, it is the aggregation of scores on the measure 
that provides us with an indication of the program's effectiveness* The measures were not 
designed to yield an accurate indication of an individual participant's status. Thus, it would 
be inappropriate to attempt to determine an individual participant's attitudes on the basis of 
the handbook's measures. The measures are relatively brief instruments designed to be 
administered without great intrusiveness. When the measures' scores are viewed in the 
aggregate, the measures can provide data of relevance to program evaluators. The data, 
however, should not be used for determining the status of individuals* 

Another point related to use of the handbook's measures concerns the potential reactivity 
of certain measures, that is, the likelihood that if the measure is used prior to the program, 
the experience of completing a measure may cause participants to react differently to the 
program than had the measure not been administered. Reactivity is more frequently 
associated with affective measures rather than cognitive measures. Thus, handbook users 
will need to be alert to the possibility that a given measure, if administered prior to the 
program, will unduly sensitize participants to an aspect of the program. 

To avoid such reactive effects, program personnel may need to divide participants into 
two subgroups so that only a portion of the participants receive any given potentially 
reactive measure* Such subgroups would not be given the same reactive measure both 
before and after the program. Rather, p.trticipants should be administered only 
post-program measures that they had not been given prior to the program* Indeed, two 
potentially reactive measures may be administered simultaneously under the conditions 
represented in Figure 1,2, where it can be seen that the pre-program performance of certain 
participants (one-half, for example) serves as a comparison for the post-program 
performance of other participants. Although a variety of data-gathering designs will be 
described in Chapter Two, the evaluate^" should employ care in using the handbook's 
measures so that they permit reasonable inferences regarding program effectiveness. 
Potential reactivity of measures should be examined when considering such designs. 
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Stress Management 
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Measure X 



Fi^re 12: Using the handbook's measures to avoid reactive effects 
(Appropriate Comparisons = } 

Technical Quality of the Handbook's Measures 

The measuring instruments to be found in Chapter Three were carefully constructed by 
an experienced test-development agency according to the guidance of prominent experts in 
the field of stress management. All of Chapter Three's assessment devices were subjected 
to small-scale tryouts, revised on the basis of those tryouts, and reviewed by stress 
management specialists. 

At the outset of this handbook-development project, it had been anticipated that all of 
the handbook's measuring instruments would be subjected to large-scale field-tests so that 
substantial empirical evidence regarding the technical quality of the measures could be 
made available to handbook users. Unfortunately, that phase of the project could not be 
completed. 

Thus, handbook users should be cautioned that, although the handbook's measures were 
developed with great care, there is currently no evidence available by which to ascertain the 
technical quality of the measures. Thus, handbook users must exercise caution in the use of 
Chapter Three*s assessment instruments. In Chapter Four, as indicated earlier, a 
desCTiption is presented of the ways which users of the handbook's measures, if they wish to 
do so, can carry out local studies regarding the technical quality of the measures that they 
find most suitable for their use. 

Specific Stress Management Concerns 

This handbook is intended to help those who design and evaluate stress management 
programs. It is not intended to transmit content dealing specifically with stress or its 
management. For those readers who wish to acquire information about stress and stress 
management, the list of reference? located at the end of this chapter contains introductory 
and advanced resources dealiug with stress per se. Nonetheless, the field of stress 
management is, in a very real sense, so distinctive that some attention to the substance of 
stress management must precede a consideration of approaches to program evaluation. As 
individuals familiar v/ith stress management recognize, the distinctiveness of the field 
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impinges in a direct manner on the way in which stress management programs must be 
evaluated. 

Definitional problems. What is stress? That question, so disarmingly simple, has proved 
thoroughly /exing to the numerous individuals who have devoted themselves during recent 
decades to the study of stress (Fischer, 1986). To illustrate the definitional difficulties 
associated with stress, a conference dealing with the nature of stress was sponsored by the 
National Institute of MentM Health in May 1985. Prominent stress researchers from the 
U.S. and abroad were assembled. The chief mission of the conference was to hammer out a 
generally acceptable definition of stress. At the close of the two-day conference, a number 
of participants reported that the definitional picture was essentially no more clear than it 
had been prior to the session. 

Such definitional disagreements serve to exemplify a series of pervasive controversies 
regarding many aspects of stress and stress management. For example, ther^ are two clearly 
divergent schools of thought regarding what causes a stressful reaction in individuals. One 
group maintains that a stressor is dominantly defined as a stimulus external to the 
individual, such as the death of a loved one (Dolurenwend, Dohrenwend, Dodson, & Shrout, 
1984). An opposing camp holds that the external stimulus is far less important than the 
manner in which the individual perceives and cognitively appraises that stimulus. For 
example, a raft-ride down a series of Whitewater river rapids may be terrifying and 
stress-inducing to one individual, yet exciting and gratifying to another. 

The stimulus-focused conception of a stressor is well represented in the attention to 
assessment scales that focus on significant life events (e.g.. Holmes & Rahe, 1967; 
Dohrenwend & Dohrenwend, 1974). Proponents of a stimulus-focused view of stress 
maintam that, although there are obvious differences in the way that individuals react to 
major life events, the dominant factor in those reactions is associated with the stimulus 
itself, not how the stimulus is perceived. Thus, divorce, the death of a spouse, or being fired 
from one's job are regarded as such powerful events that they produce, without exception, 
stressful responses in people. 

On the other hand, those who emphasize the role of cognitive appraisal in how one deals 
with potential stressors (e.g., Lazarus & Folkman, 1984) believe that the way in which we 
perceive external stimuli is most influential in how we react to those stimuli. In essence, 
proponents of this view argue for a perception-focused conception of stressors. To illustrate, 
the death of a spouse who has been suffering from an incurable disease might be perceived 
by the surviving spouse not as a tragic loss but as a blessing. Retirement, generally regarded 
as a stress-inducing life event, might be perceived by some as overdue relief from an 
aversivejob. 

Clearly, there are middle positions taken on this pivotal issue by many stress specialists. 
Nonetheless, the disagreement between those who advocate a stimulus-focused versus a 
perception-focused conception of stressors illustrates but one of a sizeable number of 
disagreements in the emerging field of stress management. These disagreements often 
create genuine difficulties for those who wish to evaluate the effectiveness of stress 
management programs. 

15 
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From the perspective of those who attempt to measure the effects of a stress 
management program, the percej-tion- focused view of stressors poses a number of problems 
that must be dealt with directly. If one subscribes to such a view, for example, then any 
self-report measures used must allow indivi^^iuals to teli us how they personally perceive 
potential stressors and how they react to those that are perceived to be actual stressors. If 
stressors exist chiefly in the eyes of the beholder, then one must analyze potential stressors 
through the beholder's eyes. 

On the other hand, if one subscribes to the overriding importance of external stimuli in 
the creation of stress, for example, by focusing on significant life events, what sort of role 
should stress management programs have in altering the life events that an individual 
encounters? No stress management program, for example, can claim that it will actually 
decrease the number of negative life events that its participants will experience. People do, 
after all, lose jobs and suffer serious illness. Thus, it would seem that when evaluating stress 
management programs based on a stimulus-focused conception of stressors, evaluators must 
attend chiefly to the ways in which participants cope with externally created stressors that, in 
many instances, they will be unable to alter. 

Because of disagreement in the field regarding the nature of a stressor, in this volume a 
somewhat middle position wiil be taken. More specifically, the term stressor will be used to 
describe a stimulus likely to produce stress or a physical reaction. Stress is defined as a 
negative affective respon:.e to stressors such as feelings of anxiety. A physical reaction is a 
physiologic response such as an increased heart rate. Note that the definition of a stressor 
hinges on those stimuli likely to result in stress or a physical reaction. Therefore, a stimulus 
might, because of its likelihood of promoting stress or a physical reaction, be regarded a 
stressor. Even with such stimuli, however, it is recognized that for a given individual the 
stimulus in question may not function as an actual stressor. 

Stress management programs to be evaluated There are numerous sorts of stress 
management programs currently being offered. Some of these programs are highly 
particularized, dealing with ihe management of stress arising from specific categories of 
stressors. Illustrative of such programs are those provided for participants who are dealing 
with (1) terminal illness of a loved one, (2) their own terminal illness, (3) divorce, or (4) the 
aftermath of rape. 

In contrast, there are general stress management programs offered which, in a period of 
several weeks, attempt to provide participants with a series of general techm'ques to deal 
with routine types of stressors. Often these are corporate programs concerned with job- 
related stress (Gherman, 1981). Although the theoretical orientations of such 
general-purpose programs vary considerably, they are all designed to promote participants' 
stress management skills. 

Because of the substantial particularization of special-focus stress management 
programs, and the resultant difficulties of creating appropriately specific assessment devices, 
the measures in this handbook were developed with general-purpose stress management 
programs in mind. Those who are attempting to evaluate more specialized stress 
management programs may, of course, find that the measures in the handbook suit some of 
their assessment needs. If more particularized measures are needed, perhaps such measures 
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can be developed by program evaluators along the lines of the assessment devices contained 
in the handbook. In essence, because a "serve-the-greatcst-demand strategy was adopted 
in the development of the handbook's measures, evaluators of special-focus stress 
management programs may need to modify the current handbook's measures. 

Proximal versus distal outcomes. Most stress management programs are relatively brief 
endeavors. Although the programs may extend over a period of several months, the actual 
number of hours constituting a stress management program is often extremely small. It is 
important, therefore, for designers and evaluators of stress management programs to focus 
on the sorts of realistic outcomes apt to be achieved by the stress management program 
under consideration. 

One distinction of utility in this regard is the difference between proximal and distal 
program outcomes. Proximal outcomes are those post-program effects apt to be directly 
promoted as an immediate consequence of the program. Thus, if a six-week program on 
stress-management emphasizes the use of certain coping techniques, the evaluation of that 
program might assess participants' proximal (1) knowledge of, (2) attitudes toward, or (3) 
actual use of the coping techniques within a period of several weeks following the program's 
conclusion. 

More distal outcomes, such as participants' subsequent ability to deal with life's stressors, 
while doubtlessly of considerable interest to program personnel, may represent aspirations 
too ambitious for the typical stress management program. Only stress management 
prograciS of substantial intensity and duration could realistically be expected to promote 
long-range, distal outcomes that are detectable. 

Stress management in evolution. More so than most fields in health education and health 
promotion, the stress management field is in its early stages. With emerging specializations, 
it is often tempting to move as rapidly as possible toward agreements so that common 
understandings may move the field forward more rapidly. 

Yet, as several observers (e.g., Janis, I.L., 1986) have noted, the dangers of prematurely 
foreclosing the delate over key stress-related issues may do the field more harm than good. 
It appears, then, that even in the absence of substantial agreement over many issues, those 
involved in stress management evaluation may still make reasonable progress. 

Just as importantly, even with their less than abundant store of empirical knowledge, 
stress management specialists can devise programs that will prove genuin.'ily useful to the 
participants served. Such programs can be beneficial // they are effective. How to use this 
handbook in evaluating the effectiveness of such programs is the focus of the next chapter. 
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Essentials of Program Evaluation 
for Health Educators 



Education programs are intended to help people. Pubil': school programs, for example, 
are intended to help youngsters acquire the skills and kno vledge that they will need as 
adults. Similarly, health education programs are intended to p/omote participants' adoption 
of beneficial health-related behaviors. Yet, even though an education program might have 
been well intentioned, how do we know that the goals of .he program were realized? 
Moreover, if a program is not meeting its goals, how can tie piogram be made more 
effective? 

Such questions constitute the core of program evaluation. In esseu -^e, evaluators want to 
discover whether a program has worked effectively and, if not, how it can be made more 
effective. When evaluation is used to improve programs, it can make a significant 
contribution to the well-being of program participants and, potentially, to the community at 
large. 

In this chapter, the nature ci program evaluation will be considered as it relates to health 
education programs. The following topics will be discussed: 

• Focusing the Evaluation 

• Rights of Participants 

• Selecting Appropriate Measures 

• When to Administer Measures 

• Data-Gathering Design Options 

• Sampling Considerations for Data Collection 

• Data Analysis 

• Reporting Results 

The purpose of this chapter is not to promote a particular evaluation model for health 
education programs. Rather, the chapter deals with considerations central to any evaluation 
effort. It is hoped that evaluators* of stress management programs will be able to apply the 
chapter's contents to their endeavors. 

Focusing the Evaluation 

The results of a program evaluation can be used to improve decisions about programs. 
Anyone setting out to evaluate a health education program, therefore, should focus the 
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Sometimes a program evaluation will be conducted by an individual not affiliated with the program 
itself -an individual formally designated as a program evaluator. More frequently, however, an evaluation 
will be carried out by the personnel who are actually operating the program. Whenever the term 
"evaluator" is used in this handbook, it will refer both to the evaluator-speciallst and to the program staff 
member serving as evaluator. 
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evaluation on the decisions that are likely to be made about the program, either while the 
program is being implemented or when it is concluded. In ether words, if evaluators know 
what decisions are apt to be faced by those who will u:e the evaluation's results, then 
information bearing on those decisions should, if possible, be collected during the 
evaluation. To determine what these decisions are, an evaluator needs to have a clear 
understanding of the purpose of the program, the specifics of the program, and the 
individuals or groups who may use the evaluation's results. Focusing the evaluation involves 
considerations such as (a) the nature and role in the evaluation of program objectives, (b) 
the summative and formative functions of evaluation, (c) the cost of the program, (d) the 
extent to which observed changes in participants will also be attributed to the program, and 
(e) the extent to which program effects will be generalizable to other situations. Each of 
these considerations is discussed below. 

Objectives and evaluation. Health education programs are designed to bring about 
worthwhile effects. Most health education programs, therefore, are organized around some 
form of program objectives that focus on such intended effects. In general, the more clearly 
these objectives are stated, the more useful they will be in carrying out an evaluation. 

One way of conducting an evaluation is to determine the extent to which a program's 
objectives have been achieved. Program designers too frequently describe their objectives in 
such ambiguous, general ways, however, that it is impossible to tell whether such loosely 
defined objectives have been attained. It is for this reason that it can be beneficial for 
evaluators to work with program personnel, prior to program implementation, to create 
program objectives that clearly describe desired post-program participant behaviors. 

Another potential pitfall when creating program objectives is the tendency to delineate a 
set of hyper-detailed objectives. Specificity does not automatically yield utility. Instead, 
decision makers can become overwhelmed by long lists of low-level, albeit behaviorally 
stated, objectives. For example, a program objective that participants be able to identify 
noise as a stressor is going to lead down a path toward numerous small-scope objectives. 
Recent thinking regarding instructional objectives suggests that program objectives, while 
still measurable, should focus on larger, more significant types of participant post-program 
behaviors. A more significant stress-related objective, for example, might be that 
participants be able to identify stressors in commonly encountered life situations in which 
stressors are apt to be present. Today's health education programs, rather than being 
.rganized around 30 miniscule (and, therefore, potentially trivial) objectives, might better 
be focused on a half-dozen more general, but still measurable, program objectives. 

Most evaluators agree, however, that there is substantially more to program evaluation 
than merely determining whether a program's objectives have been achieved. For example, 
there may be effects of the program that were not a-^ticipated in the program's stated 
objectives. Evaluators need to be attentive not only to the effects of a program that were 
anticipated but also to any unforeseen program effects. 

Summative and formative functions. Summative evaluation addresses the question of 
whether a program, in its complete and final form, is effective. The decisions associated with 
the summative evaluation are essentially go/no-go decisions, such as whether to continue a 
health education program or, perhaps, whether to disseminate the program more widely. 
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Formative evocation addresses q^ -^stions associated mth improving a program that is 
"under-development," that is^ still modifiable. The decisions associated with formative 
evaluaiior focus on ways to improve particular parts of the program. Formative evaluation is 
an ongoing endeavor conducted as the program is designed, installed, and maintained. 
Whereas summative evaluation'? mission is to provide a final judgment about a program's 
overall merit, formative evaluation's mission is to bolster a program's quality on a 
continuing basis. The effective formative evaluator functions less as an external judge and 
more as a collaborating member of the program team. The formative evaluator's task is to 
monitor the program so that it can be improved. 

Almost all programs are, at least to some degree, modifiable. Hence, only in rare cases 
do evaluators appraise a health education program in its complete and final form. One such 
instance might involve a materials-based stress management program. For example, if the 
program were found to be effective via a suminative evaluation, a commercial publisher 
would distribute the program's materials nationally. In most cases, however, health 
education programs can be modified and improved. Thur>, a formative, 
improvement-oriented evaluation can be carried out for most health education programs. 

Cost analysis considerations. Program evaluators are often so concerned about detecting 
the effects of programs that they fail to consider the costs of those effects. Yet 
decision-makers need inform^^tfon regarding not only the effects of a program, but also the 
resources required to achieve those results. For this reason, program evaluators should 
carefully isolate and communicate the relative costs of programs. For example, information 
should be collected tiat can show how much Program A costs to produce a given result 
compared to the cost of Program B to produce a comparable result. Judgments about a 
program's impact without considerations regarding its costs are potentially superficial. In 
recent years there has been much attention to cost-analysis strategies. Although 
consideration of those procedures is beyond the scope of this handbook, serious evaluators 
of health education programs would do well to delve more deeply into cost-analysis 
procedures.* 

Attributing observed changes to the program. Characteristically, an evaluation seeks to 
determine whether individuals have changed as a result of their participation in a program. 
The key issue is whether pre-program to post-program changes in the status of participants 
are attributable to the program itself or to other extraneous factors. Examples of extraneous 
factors are participants' maturation, their familiarity with the measures used in the 
evaluation, or their reactions to non-program events such as a health-related, mass media 
campaign. This issue revolves around the evaluator's ability to properly infer that the 
program itself caused any observed changes in participants. Technicall}, the degree to which 
evaluators can validly infer that a program caused a set of observed changes is referred to as 
the internal validity of the evaluation study. Ideally, an evaluation's data-gathering design 
should help to rule out explanations other than the program itself for observed changes. 
(Data-gathering design options are discussed later in this chapter.) If evaluators are unable 



For additional information about cost analysis approaches, sec Annotated Bibliography Nos. 1, 28, and 29. 
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to attribute observed changes to the program, they wall have difficulty in deterraining 
program quality. 

Generalizing program effects. A related issue is the extent to wiiich the findings of an 
evaluation smdy can be generalized to other situations. The issue here is whether similar 
results would be expected, for example, with a different group of participants, slight 
variations in the program, or changes in program personnel. The degree to which the results 
of an evaluation study can be generalized elsewhere is technically described as the study's 
external validity. 

If evaluations are generalizable, they can provide useful information to (a) program 
personnel regarding the range of conditions under which the program is effective and (b) 
other health educators who may wish to adopt an already "evaluated'* health education 
program. A stress-management program that works well in one setting may provide helpful 
guidelines for those wishing to operate other stress programs. Typically, however, a local 
evaluation should be conducted once the program has been adopted. 

It is important to distinguish between a program's causative power and the program's 
generalizability, because different information may be required to establish each factor. 
Procedures that limit the number of extraneous variables in the evaluation (e.g., including 
only males) inaease internal validity but, at the same time, limit generalizability. Evaluators 
must try to balance the problems associated with threats to internal and external validity by 
selecting a data-gathering design that best addresses the information needs of program 
personnel as well as of those external to the program who may be interested in adoptinj^ the 
program elsewhere. 

Rights of Participants 

Health education programs are designed to improve individuals' health and well-being. 
When such programs are evaluated, therefore, the focus is typically on a program's impact 
on human beings. Some evaluators, however, become so caught up with the importance of 
appraising a health education program that they overlook the rights of the individuals who 
take part in the evaluation. Two important rights are those of informed consent and 
confidentiality. 

Informed corjent. Evaluators, just as researchers, should be guided by a profound respect 
for human dignity. Therefore, they should not engage in evaluative activities that in any way 
demean participants. Prominent among the considerations that should guide evaluators is 
the concept of informed consent. Informed consent requires that an evaluator secure, in 
advance of the study, permission from the participants in an investigation to gather data 
from them. This consent is obtained after the potential participants have learned about the 
nature of the investigation and what their role would be, because that information may 
influence their decision to participate. Informed consent eliminates the possibility of making 
individuals unknowingly serve as subjects in an evaluation. 



For additional information about Iniernal and external validity li^i^aes, sec Annotated Bibiliography Nos. 8, 
11, 12, and 15. 
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Two different approaches to securing informed consent have been employed by program 
evaluators. The first of these, active informed consent, obliges an evaluator to obtain, in 
writing, a statement from each participant indicating that the individual is willing to 
participate in the evaluation. The significant aspects of the evaluation must be described in 
the written permission form so that potential participants are fully informed when they give 
their consent. 

An evaluator using the second approach, ;7as'5/ve informed consenty supplies descriptions 
of the evaluation's essentials to all program participants and provides them an opportunity 
to register, in writing, their unwillingness to participate in the study. In other words, when a 
passive informed consent approach is used, participants return the forms supplied to them 
only if they are not willing to participate in the evaluation study. Of the tv/o approaches, the 
active informed consent strategy typically results in fewer participants because those 
individuals who do not provide consent forms must be excluded from the study. Because 
evaluators who conduct studies involving school-age children are obliged to secure informed 
consent from underage participants' parents or guardians, a passive informed consent 
strategy is often adopted due to the difficulty of securing active informed consent from 
individuals who are not participating in the program themselves. 

Procedures for developing forms for both of th^se approaches to securing informed 
consent are described in Appendix B. The actual forms to be used in an evaluation would 
need to be created so that they are specifically relevant to the program involved. 

Confidentiality. Another consideration when dealing with human subjects is the 
confidentiality of all information gathered during an evaluation. Because the evaluator is not 
concerned with an appraisal of individual participants but, rather, with gauging the 
effective of a health education program, ensuring participant confidentiality usually 
poses no problem. Evaluators must, however, devise protective safeguards, such as 
anonymous completion of forms and carefial handling of data, to ensure both the 
appearance and reality of confidentiality.* 

Selecting Appropriate Measures 

Although there are various approaches to program evaluation, almost all share one 
common feature, namely, the systematic gathering of evidence regarding a program's 
effects. To secure evidence of program effects, evaluators usually employ measurement 
instruments. Some instruments, however, are far more suitable for assessing a program's 
effects than others. 

Criterion-referen^ :d measurement. For more than two decades, educational measurement 
specialists have directed increasing attention toward an emerging form of assessment known 
as criterion-referenced measurement. In comparison to norm-referenced measurement, 
which attempts to ascertain an examinee's status in relation to the status of other examinees, 
criterion-referenced measurement attempts to ascertain an examinee's status in relation to a 



* For additional Information about the rights of human subjects and the ethics of evaluation, see Annotated 
Bibllobraphy Nos. 2, 26, and 38. 
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criterion-referenced measurement attempts to ascertain an examinee's status in relation to a 
clearly defined set of behaviors. The essence of a criterion-referenced instrument is the 
clarity with which its accompanying descriptive materials explain what is being measured. 
Because norm-referenced instruments emphasize relative comparisons among examinees, 
they often do not provide a clear description of exactly what it is they are assessing. In 
contrast, criterion-referenced instruments are absolute measures, designed to determine 
exactly what it is that examinees can or cannot do, without reference to the performance of 
other examinees. Thus, criterion-referenced tests provide a clearer description of what they 
are measuring. 

It is the clarity regarding what is being assessed that renders criterion-referenced 
measures ideal for the evaluation of health education programs. Consistent with the mission 
of providing useful information for decision-makers, criterion-referenced instruments 
describe the precise nature of what is being measured. Hence, when criterion-referenced 
measures are used to gather evidence in program evaluations, decision makers can 
accurately interpret the evidence being supplied.* 

Attributes of well-constructed measures. All instruments, whether norm-referenced or 
criterion-referenced, should measure what they are measuring with consistency. The 
consistency with which an instrument measKres is known as its reliability.** There are 
several different indices that can be computed to reflect an instrument's reliability. The kind 
of reliability data needed to appraise a measure fux possible use in an evaluation study 
should be consonant with the way the measure will be used in that study. If a measure is to 
be used on a test-retest basis, for example, then information about that type of reliability is 
germane. If ahernate forms of a test are to be used, for instance, in a pretest-posttest 
situation, then evidence :>hould be available regarding alternate-forms reliability so that the 
evaluator can determine whether or not the two different forms are sufficiently equivalent. 

It should be noted that when a health education program is being evaluated, attention 
should be directed to the impact of the program on a group of participants. Thus, the 
consistency to be sought when measurement instruments are used for program evaluation is 
consistency for a group of participants' scores. When dealing with individual participants, 
the measures must yield individual or diagnostic consistency. 

A second critical attribute of a properly constructed measure is that it yields scores from 
which valid inferences can be drawn. An instrument is often said to be valid "if it measures 
what it purports to measure." Such a statement, however, is technically in error. Tests 
themselves are never valid or invalid. Rather, it is the interpretations made from test scores 
that are valid or invalid. 



* For additional Information about the nature and development of criterlon rcfercnced measures, see 
Annotated Bibligraphy Nos. 6, 24, and 35. 

** For Information about determining the rcliabilit> of measuring inslrumcnls, sec Annotated Bibliograph) 
Nos. 3, 18, 19, 23, 27, and 35. 
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There are several types of validity evidence, each yielding somewhat different but 
conceptually related indications about our ability to make valid inferences from a measure. 
Evidence of validity is, in the opinion of most measurement specialists, the most important 
consideration in judging the adequacy of measurement instruments. Program evaluators 
should make sure they are knowledgeable about methods of securing validity evidence.* 

A final consideration in appraising the quality of measures used for program evaluation 
deals with the presence of bias in the assessment devices. During the past decade, 
measurement specialists have become particularly aware that many educational assessment 
devices contain items biased against particular subgroups, such as ethnic minorities or 
women. An example of a biased test item would be a knowledge question that, because of 
peculiarities in its content or wording, is more difficult for women to understand and answer 
correctly than it is for men, even though the men and women have an equivalent amount of 
knowledge regarding the particular concept being tested. 

Another type of bias that can adversely influence examinee performance arises when test 
items are offensive to particular groups of individuals. For example, if a test item includes 
content that is seen to be derisive to members of particular ethnic groups, then examinees 
from those groups are not apt to perform at ti;eir best on the item. Their warranted agitation 
over the offensive content is likely to interfere with their responses to that item as well as to 
subsequent items. 

There are now available both judgmental and empirical techniques for detecting the 
presence of biased items. These approaches should be used to identify, then eradicate, bias 
in a measure's items.** 

Finally, it is important to note that any given instrument may not possess all of the 
qualities discussed above. Often evaluators must choose among measures that embody some 
but not all of the elements described here, that is, (a) descriptive clarity, (b) reliability, (c) 
validity, and (d) absence of bias. Another important point is that merely because a measure 
is labeled in a particular way, for example, as criterion-referenced or as non-biased, that 
does not automatically indicate that it is of sufficient quality' to be used in evaluating a 
health e Jucation program. Scrutiny of all aspects of the measure's quality is requisite. 

W^en 1 .0 Administer Measures 

^ecisnns regarding when to administer measures depend on the data-gathering design 
selected. Conceivably, there are four temporal periods during which it may be useful to 
obtain evaluative information about participants of health education programs. There may 
also be reasons for repeated measurement during some of these periods. These periods are 
depicted in Figure 2.1. 

Pretests. Often it is useful to have information about participants prior to their starting 
th** program. Such information, typically referred to as pretest data, may be used to identify 



* For information about obtaining validity evidence regarding measuring Instruments, see Annotated 
Bibliography Nos. 3, 18, 19, 23, 27, and 35. 

* * For mformation about methods for avoiding test bias, sec Annotated Bibliography Nos. 7 and 33. 
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Figure 2.1: Possible measurement times in program evaluation studies 

participant needs so that instruction can be targeted directly at those areas. In addition, 
pretest data can be compared with data collected at the end of a program. Such a 
comparison can provide a measure of program impact. 

En route tests. Measures can also be administered during a program to secure current 
readings on the status of participants. For purposes of formative evaluation, en route data 
can be used to redirect resources during the program by providing program personnel with 
ongoing status-checks on participants' progress. Thus, en route tests may be even more 
useful than tests administered at the end of the program, because en route measurement 
provides information while there is still time for piOgram personnel to act on it. This type of 
assessment is most appropriate for programs of long duration (e.g., several months or 
more), 

Posttests. Measures are commonly administered following a program. The data from 
posttests can be compared with pretest data to examine changes in participants from the 
beginning to the end of the program. Participants* posttest performance can also be 
contrasted with posttest scores from participants in other programs. In addition, posttest 
data provide an indication of the absolute status of participants on the variables of interest 
at the completion of the program. 

Delayed posttests. Data from delayed or follow-up posttests are often as important or 
more important than immediate posttest data in evaluating a healtl education program. 
Delayed posttest data might be secured, for example, several months after a program's 
conclusion. Far too frequently data collection efforts are limited to those times when 
measurement is most convenient. Uhimately, however, health educators should be 
interested in effecting long-term, rather than short-term, behavioral, affective, and cognitive 
changes. It is nearly impossible to infer such long-term changes on the basis of information 
gathered solely at the end of a program. As indicated in Chapter One, many of the desired 
changes in participants of stress management programs represent long-term rather than 
short-term objectives. For most health education programs, some follow-up measurement, 
perhaps repeated, is usually warranted. 

Clearly, it is not sensible to administer all measures at all time periods. Evaluators, in 
collaboration with program personnel and other interested parties, need to select a 
measurement scheme that focuses on the most appropriate times for gathering data. Just as 
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it is desirable to avoid administering an excessive number of different measures, it is also 
necessary to avoid an excessive number of admim*strations. It may be useful to administer 
certain measures (for example, a brief behavioral self-report measure) on a continuing 
basis; other more time-consuming measures might be administered less frequently. 
Decisions about when to administer measures should be guided by common sense, 
attentiveness to participants' feelings, the efficient use of resources, and any conventional 
expectations, such as when a delayed posttest is ordinarily given. 

Data-Gathering Design Options 

It is sometimes thought that program evaluations must include complicated and 
elaborate data-gathering designs in order to yield decisive and compelling data. This is 
simply not the case. Program personnel and evaluators should try to conduct evaluation 
studies and gather data in such a way that the ambiguity of results can be reduced to a 
minimum. That L, evaluations must attempt to determine whether a program works and 
what makes it work or what prevents it from working. Data-gathering designs serve as the 
means to this end by setting forth the procedures to be used in exploring the nature and 
impact of a program. 

The data-gathering design that an evaluator chooses for an evaluation will determine the 
inferences the evaluator can make about a program's overall impact on participants and the 
effectiveness of its various components. To select the best designs for evaluation studies, 
evaluators must have a broad knowledge of the available data-gathering design alternatives 
and the strengths and weaknesses associated with each. Evaluators must also work closely 
with program staff to determine what decisions are at issue regarding the program. No 
evaluation study v/ill be perfect; every evaluation leaves some questions unanswered. 
Evaluators need to be clear regarding what they have learned about a program and the 
degree of certainty associated with their findings, and they must convey this information to 
appropriate audiences. 

An important concept related to data-gathering designs is randomization. Randomized 
selection and assignment are described below, followed by brief descriptions of the most 
common data-gathering designs available for evaluators of health education programs. 

Randomization. One techmque that can prove useful to evaluators is randomization, 
which involves the selection or assignment of participants in a nonsystematic manner, such 
as by using a table of random numbers (found in most statistics texts). A prominent 
application of randomization in program evaluation is randomized selection of subjects. This 
sort of randomization is particularly important when the evaluator wishes to generalize from 
the results of a study to a larger population. When the participants taking part in the 
program to be evaluated have been selected at random from a larger population of potential 
participants, then the evaluator can be reasonably confident that those involved in the 
evaluation will be representative of that larger population. There is less likelihood that the 
participants being studied in the evaluation are atypical, which would make it inappropriate 
to generalize the evaluation's results to the population at large. Randomized selection of 
subjects may also be useful when there are more applicants than vacancies for a program. 

Another use of randomization is to assign participants to different "treatments" or 
programs. If an evaluator wishes to compare the effects of different treatments, then the 

23 

28 




Figure 22: Randomized selection of participants from pool of potential panlcipants and 
randomized assignment of participants to programs 



evaluator wants the participants in each treatment to be as equivalent as possible. To this 
end, evaluato:^ can employ a randomized assignment procedure whereby individuals are 
randomly placed in the treatments or programs to be compared. 

The two procedures of randomized selection and randomized assignment are illustrated 
in Figure 2.2. Note that participants are randomly selected from the pool of potential 
participants, and then randomly assigned to either Program A or Program B. 

The use of randomization techniques does not necessarily create equivalent groups. For 
example, if an evaluator were to randomly assign 50 potential participants in a company's 
stress management program to treatment and no-treatment groups, it is still possible that 
one of the groups would contain individuals who, when pretested, were significantly 
different in some important aspect from those in the other group. In such instances, 
evaluators must rely on statistical procedures in an effort to compensate fo* such disparities. 
In most cases, however, use of randomization will create groups of sufficient equivalence 
that such statistical adjustments are not needed. 

In practice, program personnel often may not have the luxury of constituting groups via 
randomized selection or assignment. For example, local school board policies might require 
that all youngsters be provided with any program regarded as potentially beneficial. When 
randomization is not used, it is especially important to collect and examine descriptive data 
about participants to determine where pre-program group differences occur and to consider 
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the ways in which such differences may influence post-program data. Even if randomization 
is impossible, attempts to constitute comparison groups with individuals as equivalent as 
possible can help minimize the influence of preexisting participant differences. 

Seven different data-gathering designs of potential utility for evaluators of health 
education programs will be presented below. Each data-gathering design will be described 
and depicted schematically. Some of the major factors involved in the selection of data 
gathering designs will be addressed. 

The case-study design. Consider a sbc-month health education program aimed at 
modifying participants' knowledge about the effects of stress on health. E participants' 
knowledge were measured only at the close of the program, we could describe the 
data-gathering approach as a case-study design and represent it schematically as shown in 
Figure 2.3. 



Program » Measurement 



Figure 23: Case-Study Desi^ 

If this were the design employed in an evaluation, what could an evaluator tell about the 
program's impact on participants' knowledge? How confident would an evaluator be that 
participants' knowledge about the effects of stress was attributable to the program? 

It would be difficult, with confidence, to attribute any effects to the health education 
program. The program, indeed, may have been totally ineffectual. In fact, participants' 
post-program loiowledge might be identical to their knowledge before the program. The 
participants could be demonstrating knowledge that they brought to the program, not that 
they acquired during the program. Because we have no measure of par.ic?V/ant knowledge 
prior to the program, we can not distinguish between preexisting knowledge and knowledge 
acquired as a result of the program. Hence, with the case-study design, it may be impossible 
to determine whether the program had any impact on participants. 

Even though attributions of causality are often unwarranted, it may be possible to secure 
useful program evaluation data with such a data-gathering design. Suppose, for example, 
that a health education program is promoting a body of knowledge so advanced that few, if 
any, individuals would be familiar with it. In such a setting, one could assume that 
participants' post-program knowledge is attributable to the program's impact because 
participants would almost certainly not h?ve acquired the knowledge without the program. 
It might not be worth the resources necessary to implement a data-gathering design capable 
of conclusively demonstrating that participants began the program unfamiliar with the 
knowledge being promoted. 



For additional information about randomization, see Annotated Bibliography Nos. 8 and 25. 
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This example illustrates an important data-gathering consideration, namely, that the chief 
mission of data-gathering designs is to rule out plausible rival explanations, that is, 
explanations other than the program's impact, that might account for the post-program 
status of participants. If there is reason to believe that participants' pre-program status may 
account for their post-program status, then a daia-gathering design should be selected that 
permits the evaluator to rale out this rival explanation. 

The one-group pretest-posttest design. Now suppose that, to avoid the major shortcoming 
of the case-study design, an evaluator measures participants' behavior both before and after 
a health education program. This data-gathering approach can be described as a one-group 
pretest-posttest design and can be represented as shown in Figure 2.4. 



Measurement ► Program ► Measurement 



Figure 2.4: One-Group Pretest-Posttest Design 



Assume an evaluator uses the one-group pretest-posttest design and that the data reveal 
a substantial shift toward more desirable behaviors between the initial and the final 
measurement. Can this change in behaviors be ascribed to the program? Unfortunately, the 
evaluator cannot be sure. There are many other factors, totally unrelated to the program, 
that may have influenced participants' behaviors. For instance, if a stress management 
program emphasized the relationship between stress and depression, and at the same time a 
series of stress-related suicides received attention in the national news, such an event may 
have influenced participants' views regarding stress and suicide. Evaluators of programs that 
serve children must also consider the possible effects of maturation during the time the 
program is offered. Participants' increased maturity may cause pre-program to post-program 
shifts in behaviors. The program itself may have contributed nothii.^ to the measured shift 
of behaviors. Such extraneous factors decrease the evaluator's ability to draw defensible 
conclusions about the program's impact. 

As was true with the case-study design, however, if there are no plausible rival 
explanations for the posUest results, the one-group pretest-posttest design can be suitable 
for the task at hand. In fact, this simple yet serviceable design is often used in formative 
evaluation. 

The one-group pretest-posttest design requires measurement before as well as after a 
program. This points to a commonly accepted but often overlooked principle of effective 
program evaluation. Evaluation is most effective when it is initiated at the beginning of a 
program. If evaluators are not called in until the end of a program, they maybe hampered in 
their efforts to design a credible program evaluation. 

The nonequivalent control/comparison group design. Program evaluators can eliminate, 
some of the more common rival explanations for changes in participants' behaviors by using 
data-gathering designs in which either comparison or control groups are employed. The use 




of a control group (untreated individuals) or a comparison group (individuals receiving a 
different program) requires two groups that are assumed to be relatively similar (before the 
program) on all related variables. When using these designs, the evaluator should attempt to 
secure two groups that are as similar as possible. Because the two groups are not randomly 
assigned to the two conditions, however, they cannot be assumed to be equivalent, hence the 
design's designation as a "nonequivalent" control or comparison group design. 

In the control-group version of this design, only one of the groups is given the program to 
be evaluated; the other group is left untreated. This data-gathering design, known as the 
nonequivalent control group design, is illustrated in Figure 2.5* 



In this design, a control group (Group 2) is assessed before and after the program, but it 
never receives the program itself. Assuming that the groups were similar before the 
program, if the program participants' behaviors change while the behaviors of those in the 
control group remain the same, the evaluator can be reasonably confident that the program 
caused the change. 

The use of an untreated control group may strike some health educators as a particularly 
unsavory data-gathering ploy. After all, health educators design their programs to benefit 
participants. To withhold such programs from individuals, even for the important purpose of 
evaluating the program's effectiveness, seems downright reprehensible. Yet, the individuals 
from whom the program is withheld, that is, the members of the control group, can be given 
the pvogiam substque ntly, as soon as the evaluation study has been concluded. Also, in some 
situations there are more program applicants than can be accommodated, and, therefore, 
some prospective participants must be denied access to this program under any 
circumstances. Those who are not admitted to the program could be used as a control group^ 
and admitted to the program the next time it is offered. 

A variation of the nonequivalent control group design involves the use of a comparison 
group, that is, a group receiving a different program or a different treatment. Program 
evaluators frequently find themselves studying the quality of two or more competing 
programs- Thus, the evaluator focuses on the relative virtues of two or more different 
programs rather than on a contrast between a single program and an untreated control 
group. A schematic depiction of a nonequivalent comparison group design, in this instance 
contrasting two different programs, is presented in Figure 2.6. As indicated above, more 
than two groups can be employed when using a nonequivalent comparison group design. An 
evaluator using this design can be fairly certain that, if the groups were similar before the 
program, any differences in post-program behaviors are due to the differential impact of the 
two programs. 



Group 1: Measurement 



Program 



Measurement 



Group 2: Measurement 



Measurement 



Figure 2.5: Nonequivalent Control Group Design 
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Group 1: 


Measurement 


► Program A 


► Measurement 


Group 2: 


Measurement 


» Program B 


► Measurement 



Figure 16: Nonequivalent Comparison Group Design 



There are, however, potential problems with the nonequivalent control/comparison 
group designs. It may be that the initial measurement was reactive, A reactive measurement 
is one that, by itself or in combination with the program, influences participants' behavior. 
Attitude inventories and self-report questionnaires about behavioral practices are 
notoriously reactive. For example, a questionnaire administered before the program might 
alert participants to the importance of a desired behavior. This would heighten their 
attentiveness when the program dealt with content related to that behavior and, as a 
consequence, influence their performance on the secopd measurement. 

Moreover, measurement is expensive. Measuring the status of control groups requires 
valuable evaluation resources'. Time and money can often be better spent studying the 
program being evaluated rather than studying a no-treatment control group of little intrinsic 
interest. Health educators should not ritualistically employ control groups in their designs if 
the questions at issue can be answered without the use of untreated groups. 

The pretest'posttest control/comparison group design. There are two data-gathering designs 
that are of particular value to program evaluators if randomized assignment is possible. The 
first of these is the pretest-posttest control group design, illustrated in Figure 2.7* 



Randomized 


Group 1: 


Measurement 


— ► Program — ► Measurement 


Assignment 


Group 2: 


Measurement 


► Measurement 



Figure 2 J: Pretest-Posttest Control Group Design 



The difference between this design and the previously considered nonequivalent control 
group design is, of course, the randomized assignment of subjects to the two groups. This 
feature of the design is a particularly important one, because creation of two or more groups 
using randomized assignment is an effective way of promoting equivalence between the 
groups, especially if the number of subjects in each group is large (say, 30 or more). 
Equivalence of groups at the beginning of the program strengthens the inference that any 
differences at the conclusion of the program are due to program impact. 




Randomized 


Group 1: 


Measurement — ^ 


Program A 


— ► Measurement 


Assignment 


Group 2: 


Measurement — ► 


Program B 


— ► Measurement 



Figure 2.8: Pretest-Posttest Comparison Group Design 

By using comparison groups, that is, two or more program groups, instead of an untreated 
control group, the evaluator would be using a pretest-posttest comparison group design^ shown 
in Figure 2.8* 

Because pretests are used in both of these designs, the possibility of reactive preprogram 
measures is still present. For situations in which reactivity is of great concern, a different 
data-gathering design, described next, has much appeal. 

The posttest'Only control group design. In situations where a measure is likely to be 
reactive, the evaluator can rely on a clever data-gathering design that effectively dodges the 
reactivity problem. This posttest-only control group design is depicted in Figure 2.9. This 
design is the same as the pretest-posttei>t control group design, except that there is no 
pretest. 



Randomized 


Group 1: Program 


^ Measurement 


Assignment 


Group 2: 


Measurement 



Figure 2.9: Posttest-Only Control Group Design 



In this design, neither Group 1 nor Group 2 is pretested, but because of random 
assignment the groups can be considered equivalent prior to Group 1 receiving the 
program. Not pretesting Group 1 effectively avoids a pretest's potentially reactive effect on 
program participants. To assess the impact of the program, it is possible to contrast the 
posttest performances of Groups 1 and 2. As with the other control group designs, the 
untreated control group could be given the program the next time it is offered. 

The basic dividend of the posttest-only control group design is that by measuring an 
untreated, randomly assigned control group, the evaluator secures an estimate of how 
program participants would have responded on a pretest, but without introducing the 
potentially reactive effects of a pretest. Although the diagram for this design suggests that 
the measurements be made for both groups at the conclusion of the program, it is possible 
to measure the untreated control group earlier if that seems advisable. 




Multiple measures over time. There are certain situations in which health educators may 
wish to appraise the effects of their programs on the basis of periodic measurements, for 
example, by using regularly administered questionnaires or data that are routinely recorded. 
For instance, suppose when evaluating a "supervisor's stress awareness'' program, the 
e valuator was interested in the number of stress-related referrals a company's supervisors 
make for their employees. Assuming that such information is available from the firm's 
health records, the evaluator might study records at periodic intervals before, during, and 
after the program. By observing the frequency of referrals during different time intervals, 
the evaluator would have valuable information regarding program effects. 

A number of the most commonly used data-gathering designs have been described. 
There are other, more complex designs than those treated here.* Complexity, however, is 
rarely an asset if a simpler, more straightforward design is appropriate. 

Sampling Considerations for Data Collection 

The data-gathering requirements of an evaluation can become a burdensome intrusion 
into an ongoing health education program. Participants in a stress management program can 
become more than mildly stressed if evaluators are requiring them to complete measures 
every hour or so. Accordingly, evaluators should conduct their data-gathering activities in 
the least intrusive manner possible. One way to minimize an evaluation's intrusiveness is by 
relying on sampling techniques, such as person-sampling and item-sampling, each of which 
is described below. 

Persori'Sampling. To estimate how a large group of people would respond on a particular 
measure, it is not necessary to administer the measure to all the individuals in the group. 
Instead, a smaller group can be selected. This smaller group can be either a simple random, 
sample or a stratified random sample, that is, a sample stratified on the basis of 
program-relevant factors such as age, sex, and socioeconomic status. Assuming that the 
sample is randomly selected, the evaluator can estimate the status of the total group based 
on the responses of the sample. 

Suppose, for example, that the evaluator wants to use a measure to determine 
participants' perceived ability to manage stress. Assuming that there is a reasonably large 
number of program participants, say 50 or so, the evaluator could randomly select half of the 
participants and administer the measure to this group only. In essence, this approach allows 
the evaluator to infer how the total group of participants would score on the measure, even 
though only half of the participants completed it. Thus, it Is possible to estimate total group 
performance with only half the amount of participant time required for data-gathering. 

Using a similar sampling procedure, evaluators can administer two or more measures at 
once in the time it takes to administer one. Suppose that two measures are to be given to 
program participants. The evaluator can randomly assign one measure to half of the 
participants and the other measure to the remaining participants. Each participant needs to 



For additional information about evaluation design options, see Annotated Bibliography Nos. 8, 11, 22, 
23, and 34. 
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respond to only one measure, but the evaluator can derive defensible estimates of how all 
the participants would have responded on both instruments. 

Item sampling. In addition to sampling persons, as in the previous examples, it is also 
possible to sample items, so that different sets of items from a program evaluation measure 
are randomly selected to be administered to different persons. Using this approach, the 
evaluator gives each participant only a sample of the items on any particular measure. For 
example, suppose a program evaluator wishes to administer a 30-item test. Given 60 
participants in the program, the evaluator could divide the test into three sets of 10 items 
each and administer each set of 10 items to 20 different participants. In this way, the total 
group's performance on the whole test can be estimated. This approach to data-gathering 
requires only one-third of the time that would have been required to administer the total 
30-item test to all participants. 

Sample size. Given the relatively small number of participants in some health education 
programs, is it really appropriate to sample either persons or items? How large must groups 
be before these sampling procedures can be sensibly used? Unequivocal answers to these 
questions do not exist. Some texts on sampling provide rules of thumb for estimating the 
size of samples needed for detecting group differences in relation to the magnitude of 
differences sought and the nature of the groups being sampled. At best, though, these rules 
provide only rough estimates. It is important to recognize that the task of identifying a 
sufficiently large sample is more difficult than usually thought. 

The variability of participants' anticipated performance on the measures is the primary 
determiner of the sample size necessary. If it is expected that participants' scores on a test 
will be relatively homogeneous, a smaller number of respondents will be needed than if 
participants' scores are expected to vary widely. Thus, if on a measure of knowledge about 
the positive ways to cope with stress, for example, some of the participants are expected to 
know many techniques and others are expected to know very few, reasonably large numbers 
of participants (e.g., 20) should respond to any one item. 

Intuitively, one recognizes that when working with a very small group of program 
participants, the use of these sampling techniques is risky. For instance, if there were only 15 
participants in a program, few evaluators would try to split these participants into three 
groups of five ea^h for purposes of taking different sets of items. Even though each group 
represents one-third of the total population, there is too much likelihood that a sample of 
five individuals would not properly represent the total group. One or two atypical 
participants in a five-person group would render th^ group's average performance 
unrepresentative of how the larger group would have performed. 

It should be noted that when employing procedures such as person-sampling or 
item-sampling, an evaluator is focusing on a group of participants in the aggregate. Because 
evaluations are typically concerned with the effects of programs on groups of participants, 
the use of sampling procedures is usually appropriate. If, however, program personnel need 
individual data on all examinees, then sampling should obviously not be employed. 



For additional information about sampling procedures, see Annotated Bibliography Nos. 9 and 10. 
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Data Analysis 

A frequent question asked of an evaluator is whether a study's results are statistically 
significant. For example, could the observed changes ia program participants* knowledge or 
behavior from pretest to posttest have occurred simply by chance? Statistical tests are used 
to answer this type of question. Consideration of statistical analysis procedures, however, is 
beyond the scope of this handbook. Indeed, for those genuinely unfamiliar with statistical 
analysis, attempts to boil down such a complex subject into a few pages would constitute a 
severe stress-inducer. Thus, just a few comments will be made here regarding data analysis. 
Because there are many subtle choice-points in the statistical analysis of evaluation data, 
evaluators who are not well versed in at least the more common statistical procedures 
should probably enlist the aid of someone who is. 

There are two basic classes of statistics, namely, descriptive statistics, such as the mean, 
and inferential statistics, such as the t test. Descriptive statistics help evaluators portray a 
group's performance on a given measure. For example, an evaluator might describe a set of 
participants' scores via the mean score (the scores* central tendency) and standard deviation 
of the scores (the scores' variability). Because the mean and standard deviation are 
frequently used, program evaluators should know how to calculate and interpret them. Any 
introductory statistics book for the social sciences will serve as a reference for this 
information. Inferential statistics help evaluators determine whether an observed difference 
between pre-program and post-program scores is statistically significant, that is, whether such 
a difference could have occurred because of chance alone. If the probability is small that the 
results are due to chance, the evaluator can, with reasonable confidence, attribute the 
results 10 the program. 

Statistical significance, however, does not imply practical significance. A small difference 
between the average scores of two groups can be statistically significant, particularly when 
large numbers of participants are involved, yet be of no practical consequence whatsoever. 
Health educators will need to make sensible determinations regarding whether the 
magnitude of an observed difference, even though statistically significant, is sufficiently 
important to warrant action. In other words, although evaluators of health education 
programs should often carry out statistical significance tests, they should not be unduly 
swayed by the results of such analyses. Common sense must always be applied in 
interpreting the meaning of a statistically significant result.* 

Reporting Results 

Reporting the results of an evaluation study is a more difficult undertaking than is usually 
recognized. Considerable attention must be given to the procedures employed to report the 
results of health education program evaluations. When reporting evaluation results, as when 
focusing and planning the evaluation, the evaluator must be responsive to the needs of 
program decision-makers. A few key considerations should be kept in mind when reporting 
evaluation results. 



For additional Information about data analysis, see Annotated Bibliography Nos. 25, 36, 39, 43, and 45. 
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Evaluators must report their results to decision makers in a timely fashion. It does no 
good to deliver an evaluation report several weeks after key program decisions had to be 
made. Evaluators must also be careful to disseminate their findings to all appropriate 
audiences. If possible, an evaluator should circulate the preliminary draft of a program 
evaluation report to program personnel so that they can react to its accuracy and objectivity. 

The decision makers whom evaluators are assisting may have scant experience with 
quantitative data. As a consequence, complicated statistical presentations may be of little 
value to them. Evaluators should select data presentation procedures that will match the 
technical sophistication of the decision makers involved. In any evaluation report, there is 
nothing wrong with simple graphs or "percentage correct" tables. The more intuitively 
comprehensible the data presentation techniques, the better they are. Program evaluators 
should provide straightforward presentations of data without fearing that such approaches 
will be regarded as too elementary. Adequate technical back-up can be appended as 
necessary to the final report. 

Evaluators should not be reluctant to make speculations based upon their knowledge 
about a program, but these conjectures should be identified as such. Similarly, if any of the 
evaluation's findings are equivocal, the evaluator should inform concerned audiences of this 
fact. Honesty and objectivity are the hallmarks of effective evaluation reporting. 

In addition, because decision makers are typically busy people, evaluators should strive 
for reasonable brevity in their reports. The preparation of executive summaries to 
accompany lengthy reports is a useful practice. Voluminous evaluation reports are almost 
certainly destined to go unread. Terse, easily read reports are much more likely to make an 
impact on decision makers. 

The whole thrust of the evaluation enterprise is to facilitate better decisions. Decision 
making will not be illuminated by complex, lengthy, or otherwise incomprehensible 
presentations of evaluation results. The quality of decision making can be enhanced only if 
an evaluation's results are reported in a way that can be clearly understood.* 

Reprise 

In this chapter, a number of issues almost certain to be encountered by evaluators of 
stress management programs were considered. Because this handbook supplies a number of 
measures to be used in the evaluation process, special attention was given to the role of such 
measures in program evaluation. Evaluators desiring more detailed treatment:^ of the topics 
covered in this chapter will find appropriate sources in the Annotated Bibliography.** 



* For additional information about reporting the results of an evaluation, ^ee Annotated Bibliography Nos. 
5,23,26, and 34. 

** For additional information about program evaluation, sce Annotated Bibliography Nos. 5, 13, 15, 20, 23, 
32,41,46, 49, and 51. 
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CHAPTER THREE 

Program Evaluation Measures 
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Overview of Measures 



1 

1 

j Category 








Page 


Title 


Target Group 


Description 


No. 


Behavior 


Stress Management 


Adults 


Assesses i?^e of stress 


39 




Checklist 


Adolescents 


management 










techniques. 






How You Deal with 


Adolescents 




44 




Your Stress 


Preadolescents 






Knowledge* 


Facts About Stress 


Adults 


Assesses knowledge 


49 






Adolescents 


of the causes and 










effects of stress- 






Learning About 


Adolescer 




55 




StresjJ 


Preadolescents 








Coping with Stress 


Adults 


Assesses knowledge 


59 






Adolescents 


of stress 










management 






Ways to Lower 


Adolescents 


techniques. 


65 




Stress 


Preadolescents 




Skill 


Appropriate 


Adults 


Assesses ability to 


69 




Responses to Stress 




select appropriate 










courses of action for 






How to Lower Stress 


Adolescents 


reducing stress- 


81 






Preadolescents 






Affective 


How Will You Feel? 


Adults 


Assesses importance 


89 








of twelve quality-of- 










life factors and 










participants 










expected satisfaction 










in one yean 





* The information eligible for inclusion In the knowledge mcdiiires is provided In Appendix A as amplified 
content descriptors. 
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Category 


Title 


Target Group 


Description 


Page 
No. 


Affective 


Keeping Your Cool 

Could You Deal 
with It? 


Adults 

Adolescents 
Preadolescents 


Asse<?ses nerceived 
ability to manage 
stress. 


92 
95 




How Will You Cope? 
What Will You Do? 


Adults 
Adolescents 

Adolescents 
Preadolescents 


Assesses intention to 
use stress 

Ukjw oil woo 

management 
techniques. 


98 
102 




People in Your Life 

The People You 
Know 


Adults 

Adolescents 
Preadolescents 


Assesses perceived 

siinnort frnm ntlipr 

people. 


106 
109 




Life Satisfaction 
Inventory 

Are You Happy? 


Adults 

Adolescents 
Preadolescents 


Assesses 

narticinant's riirrent 
satisfaction with 
several quality-of-iife 
factors. 


112 
115 




Ideas About 
Decisions 


Adolescents 
Preadolescents 


Assesses belief in the 
value of careful 
decision-making. 


117 
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STRESS MANAGEMENT CHECKLIST 



This behavior measure examines the frequency with which participants have used 
a variety of stress management techniques during the recent past- After describing 
frequency of use, participants indicate how helpful each technique was in reducing 
stress. Both positive and negative techniques for managing stress are included on the 
measure. This measure is appropriate for adults and adolescents. 

If this measure seems useful, you might also want to consider the How Will You 
Cope? measure, which is an affective measure assessing participants' intention to use 
stress management techniques. 

PURPOSE 

Information regarding participants' use of coping techniques may be helpful for 
the following reasons: 

• Administration of this measure at the beginning of the 
program may provide needs assessment information. For 
example, results from this measure may indicate the need to 

(1) broaden participants' array of positive coping techniques, 

(2) increase the frequency with which they use positive rather 
than negative techmques, and/or (3) stren^hen participants' 
beliefs in the value of positive coping strategies. 

e When given at the beginning and end of a program, results will 
demonstrate changes in the frequency with which participants 
use i)ositive versus negative coping techniques, the variety of 
positive techniques used, and the perception of the 
effectiveness of using positive coping techniques on stress. 

PROCEDURES 

In most cases, this instrument should be administered both at the beginning and 
the end of the program. If the program is fairly long, (several months or more), the 
instrument may^ be given as it exists in this handbook. If the program is shorter than 
two months, it is jjossible that the program will not produce the behavior^ changes 
measured by this instrument. Short term gains in this area can be more realisticSly 
measured by Kow Will You Cope? -an affective measure found later in this chapter. 
Programs of shorter duration will still be able to use this measure for the purposes 
listed above for the beginning^ of the program. However, if such programs wish to 
measure pretest to posttest gains using this instrument, the instructioas that include 
the term decent past' should be changed to read: 

"Think back over the past few weeks'' 
and 

"In the past few weeks, when you experienced stress, how often did you . . 
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SCOraNG AND ANALYSIS 



Column 1: Frequency 

Items 2, 4, 9, 14, 16, 19, 22, 24, and 26, are commonly considered some of the 
negative ways for handling stress. For purposes of sconng, they will be labeled 
"negative items." Items 1, 3, 5, 6, 7, 8, 10, 11, 12, 13, 15, 17, 18, 20, 21, 23, and 25, are 
positive coping techniques. They will be labeled "positive items." 

Total Score for Negative Items Marked OFTEN or SOMETIMES 

1. Count the number of negative items that are marked either OFTEN or 
SOMETIMES. (Ignore any blarf-" or NO responses.) 

2. Divide this total by the number of program participants. 

EXAMPLE: Imagine that there are 10 program participants. First, count all 
the times that these individuals marked either OFTEN or SOMETIMES on 
the negative items (2, 4, 9, 14, 16, 19, 22, and 24). Let's assume the total 
number of times was 55. Then, divide 55 by 10 participants to get an average 
score of 5.5. 

Scores can range from 0-9. A low group average indicates that the ^oup of 
participants uses few negative coping techniques and a high group average mdicates 
the use of many negative techniques. 

Subtotal Scote f jr Negative Items Marked OFTEN or SOMETIMES 

1. Count only the number of negative items that are marked OFTEN on the 
negative items (2, 4, 9, 14, 16, 19, 22, 24, and 26). 

2. D ivide this total by the number of times the negative items are marked 
OFTEN or SON^TIMES. Multiply this number by 100 to determine the 
percentage. 

EXAMPLE: For the same 10 individuals used in the example above, count 
the number of times they marked OFTEN on the negative items. Let's assume 
the total was 35. Then, divide by the total number of times the 10 individuals 
marked either OFTEN or SOMETIMES on the negative items. This number 
was already determined to be 55 in the previous example. Now, divide 35 by 
55 to find out, of the negative strategies used, what percentage are used 
OFTEN. In this case, 35/55 is about 64%. Thus, of tne negative strategies 
used, 64% are used often, and 36% are used sometimes. 

Besides seeing an overall reduction in the number of negative coping strategies 
employed, program evaluators would hopefully see a decrease in the OFTEN 
percentage and an increase in the SOMEnMES percentage when the measure is 
administered at the beginm'ng and end of the program. 
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Scoring for Positive Items Marked OFTEN or SOMETIMES 



Items measuring frequency of use of positive techniques for handling stress can be 
scored in the same manner as the negative items. However, in this case, because of 
the larger number of positive items on the instrument, the {possible range of scores is 
from 0-17 with low scores indicating the use of few positive coping strategies and 
high scores indicating the use of m any positive strategies. It would be desirable to 
see a higher percentage of OFTENs being reported for the use of positive coping 
strategies. 

Note: When dealing with the scoring of positive stress management techniques, 
program evaluators should not be overly concerned about group scores that do 
not extend into the upper end of the range. It seems unlikely that even the most 
skilled positive stress management participants would use all the positive coping 
strategies listed. Rather, individual participants may find a limited number of 
positive strategies that work well for them. 



Column 2: Did it help to reduce your stress? 

Information from Column 2 indicates participants' perceptions regarding the 
effectiveness of the positive techniques that they use to cope with stress. This column 
can be scored by focusing on the positive 'terns (see paragraph one under Column 1: 
Frequency). 

Scoring: 

1. Count the number of positive items marked A LOT or A LITTLE. 

2. Divide that total by the total number of positive items marked OFTEN or 
SOMETIMES in Column 1. (Be sure not to include any responses marked 
NEVER in Column 1.) 

3. Multiply by 100 to determine a percentage. 

4. Repeat steps 1-3 with positive items marked NO. 

The percentages give the program evahiator an indication of the participants' 
perception of the effectiveness of the positive stress management techm'ques being 
used. 

EXAMPLE: For the same 10 individuals used in the examples above, count 
the number of positive items (1, 3, 5, 6, 7, 8, 10, 11, 12, 13, 15, 17, 18, 20, 21, 
23, and 25) that are marked A LOT or A LITTLE. Let's assume that this 
total was 115. Now, count ihe number of positive items marled OFTEN or 
SOMETIMES in Column 1; let's say 160. Then, divide the Column 2 total by 
the Column 1 tot^ ' In this case 115/160 is about .72 or 72%. Calculating the 
positive items marked NO using the same method would yield cd)out 28%. 
Thus, 72% of the participants felt that they were helped to some degree by the 
positive coping strategies they used; 28%feh they were not helped. 
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STRESS MANAGEMENl^ CHECKLIST 



Listed below are things people may do when they experience stress. 
Think back over the recent past Think about how you coped with 
any problems or stressful situations that you faced in the recent 
past. Put a flieck in Column 1 to show how often you did each thing 
when you experienced stress. Put a check in Column 2 to show 
whether or not the action helped to reduce your stress. If you 
checked NEVER in Column 1, you do not need to answer Column 2 
for that item. 



In the recent past, when you 
experienced stress, how 
frequently did you . . . 



1. tell yourself positive things? 

2. drink more coffee? 

3. try to figure out what upset 
you about the problem? 

4. eat more? 

5. find humor in the situation? 

6. use relaxation techniques? 

7. exercise more? 

8. talk about the situation 
with the people who were 
involved? 

9. smoke more? 

10, think about the positive 
side of the situation? 

IL think through how you 
would handle the situation? 

12. try to get more information 
about the situation? 



Column 1 



Some- 
Often times Never 



Column 2 



Did it help to 

reduce your 
stress? 



A 
Lot 



A 
Little 



No 
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Stress Management Qieckiist, p. 2 



Column 1 



In the recent past, when you 
experienced stress, how 
frequently did you • • . 



Some- 
Often times Never 



13. make a plan of action 
and follow it? 

14. drink more alcohol? 

15. accept the situation if 
nothing could be done 
to change it? 

16. take drugs or non-prescription 
medications? 

17. take things one step at 
a time? 

18. talk about your feelings 
with your family or 
friends? 

19. take your frustration out 
on other people? 

20. learn new skills to help 
you deal with the situation? 

21 . get advice from someone 
who could help you? 

22. take the blame for a 
problem that wasn't 
your fault? 

23. do other things for awhile 
to give your mind a rest 
from the situation? 

24. keep your feelings to 
yourself? 

25. consider several alternatives 
for handling the situation? 

26. sleep more? 



Column 2 



Did it help to 

reduceyour 
stress? 



A 

Lot 



A 
Little 



No 



HOW YOU DEAL WITH YOUR STRESS 



This behavior measure examines the frequency with which participants have used 
a variety of stress management techniques during the recent past. After describing 
frequency of use, participants indicate how helpfol each technique was in reducing 
stress. Both positive and negative techniques for managing stress are included on the 
measure. This measure is appropriate for adolescents and preadolescents. 

If this measure seems useful, you might also want to consider the What Will You 
Do? measure, which is an affective measure assessing participants' intention to use 
stress management techniques. 

PURPOSE 

Information regarding participants' use of coping techniques may be helpful for 
the following reasons: 

m Administration of this measure at the beginning^ of the 
program may provide needs assessment information. For 
example, results from this measure may indicate the need to 

(1) broaden participants' array of positive coping techniques, 

(2) increase the freq^uency with which they use positive rather 
than negative techniques, and/or (3) strengthen participants' 
beliefs in the value of positive coping strategies. 

e When given at the beginning and end of a program, results will 
demonstrate changes in the frequency with which participants 
use positive versus negative coping techniques, the variety of 
positive techniques used, and the perception of the 
effectiveness of using positive coping techniques on stress. 

PROCEDURES 

In most cases, this instrument should be administered both at the beginning and 
the end of the program. If the program is fairly long, (several months or more), the 
instrument may^ be given as it exists in this handbook. If the program is shorter than 
two months, it is possible that the program will not produce the behavioral changes 
measured by this instrument. Short term gains in this area can be more realistically 
measured by What Will You Do?— an affective measure found later in this chapter. 
Programs of shorter duration will still be able to use this measure for the purposes 
listed above for the beginning of the program. However, if such programs wish to 
measure pretest to posttest gains using this instrument, the instructions that include 
the term ''recent pasf should be changed to read: 

"Think back over the past few weeks.'' 
and 

"In the past few weeks, when you experienced stress, how often did you . . 
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SCORING AND ANALYSIS 



Column 1: Frequency 

Items 6, 10, 13, 15, 18, and 19 are commonly considered some of the negative ways 
for handling stress. For purposes of scoring, they will be labeled "negative items/* 
The remaining items are positive coping techniques. They will be labeled "positive 
items." 



Total Score for Negative Items Marked A LOT or SOMETIMES 

1. Count the number of negative items that are marked either A LOT or 
SOMETIMES. (Ignore any blank or NEVER responses.) 

? Divide this total by the number of program participants. 

EXAMPLE: Imagine that there are 10 program participants. First, count all 
the times that theiJ individuals marked either A LOT or SOMETIMES on the 
negative items (6, 10, 13, 15, 18, and 19). Let's assume the total number of 
times was 32. Then, divide 32 by 10 participants to get an average score of 3.2. 

Scores can range from 0-6 with low numbers indicating a group of participants 
who use few negative coping techniques and high numbers indicating the use of many 
negative techniques. 

Subtotal Score for Negative Items Marked A LOT or SOMETIMES 

1. Count only the number 0£ negative items (6, 10, 13, 15, 18, and 19) that are 
marked A LOT. 

2. Divide this total by the number of times the negative items were marked A 
LOT or SOMETIMES. Muhiply this number times 100 to obtain a 
percentage. 

EXAMPLE: For the same 10 individuals used in the example above, count 
the number of times they marked A LOT on the negative iteins. Let's assume 
the total was 11. Then, divide by the total number of times the 10 individuals 
marked either A LOT or SOMETIMES on the negative items. This number 
was already determined to be 32 in the previous example. Thus, divide 11 by 
32 to find out, of the negative strategies used, what percent are used A LOT. 
In this case, 11/32 is about 34%. Thus, of negative strategies used, 34% are 
used often, and 66% are used sometimes. 

Besides seeing overall reduction in the number of negative coping strategies 
employed at the beginning of the program, program evaluators would hopefully see a 
decrease in the A LOT percentage and an mcrease in the SOMETIMES percentage 
when the measure is administered at the begimiing and end of the program. 

Scoring for Positive Items Marked A LOT or SOMETIMES 

Items measuring frequency of use of positive techniques for handling stress can be 
scored in the same manner as the negative items. However, in this case, because of 

4^8 



the larger number of positive items on the instrument, the possible range of scores is 
from 0-15 with low scores indicating the use of few positive coping strategies and 
high numbers indicating the use of many positive strategies. The determination of 
percentages for the SOMETIMES and A LOT columns can also be done with the 
positive strategies. It would be desirable to see a higher percenu.^e of A LOTs being 
reported for the use of positive coping strategies. 

Note: When dealing with the scoring of positive stress management techniques, 
program evaluators should not be overly concerned about group scores that do 
not extend into the upper end of the range. It seems unlikely that even the most 
skilled positive stress management participants would use all the positive coping 
strategies listed. Rather, individual participants may find several positive 
strategies that work well for them. 



Column 2: Did it help to reduce your stress? 

Information from Column 2 indicates participants* perceptions regarding the 
effectiveness of the positive techniques that they use to cope mth stress. This column 
can be scored by focusing on the positive items (see paragraph one under Column 1: 
Frequency). 

Scoring: 

1. Count the number of positive items marked A LOT or A LITTLE. 

2. Divide that total by the total number of positive items marked A LOT or 
SOMETIMES in Column 1. (Be sure not to include any responses marked 
NEVER in Column 1.) 

3. Multiply by 100 to obtain a percentage. 

4. Repeat steps 1-3 with positive items marked NO. 

The percentages give the program evaiuator an indication of the participants' 
perception of the effectiveness ofthe positive stress management techniques being 
used. 

EXAMPLE: For the same 10 individuals used in the examples above, count 
the number of positive items (any item that is not 6, 10, 13, 15, 18, and 19) 
that they marked A LOT or A LITTLE. Let's assume that this total was 95. 
Now, count the number of positive items marked A LOT or SOMETIMES in 
Column 1; let's say 140. Then, divide the Column 2 total by the Column 1 
total. In this case 95/140 is about .68 or 68%. Thus, 68% of the participants 
felt that they were helped to some degree by the positive coping strategies they 
used. The percentage of positive items marked NO could be obtained using 
the same method. 
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HOW YOU DEAL WITH YOUR STRESS 



Listed below are things people may do when they experience stress. 
Think back over the recent past. Think about how you coped with 
any problems or stressful situations that you faced in the recent 
past. Put a check in Column 1 to show how often you did each thing 
when you experienced stress. Put a check in Column 2 to show 
whether or not the action helped to reduce your stress. If you 
checked NEVER in Column 1, you do not need to answer Column 2 
for that item. 



In the recent pasty when you 
felt stress, how often did 
you . . . 



!• think about the good things 
in your life? 

2. try to figure out what upsel 
you about the problem? 

3, use relaxation skills? 

4, try to see the good side of 
the situation? 

5. exercise more? 

6. smoke cigarettes? 

7, think about how you would 
handle the situation? 

8, try to get more information 
about the situation? 

9. make a plan of action and 
follow it? 

10. have a drink (such as beer or 
wine)? 

1 L accept the situation if 
nothing could be done to 
change it? 



Column 1 



A Some- 
Lot times Never 



() 



) () 



Column 2 



Did it help to 
reduce your 
stress? 



A 

Lot 



A 
Little 



() () 



No 

() 

() 
() 

() 
() 
() 

() 

() 

() 

() 

() 
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Column 1 



In the recent pasty when you 
felt stress, how often did 
you • • • 



12. do other things for awhile to 
get your mind off the 
situation? 

13. get high? 

14. talk about your feelings with 
your family? 

15. take your anger out on other 
people? 

16. learn new skills to help you 
deal with the situation? 

17. ask others for help if you 
needed it? 

18. take the blame for a problem 
that wasn't your fault? 

19. keep your feelings to 
yourself? 

20. think. about some different 
ways to handle the situation? 

21. talk about your feelings with 
your friends? 



A Some- 
Lot times Never 



Column 2 



Did it help to 

reduce your 
stress? 



A 

Lot 



A 
Little 



No 
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FACTS ABOUT STRESS 
(FORMS A &B) 



This knowledge measure examines what participants know about the types of 
activities and events that cause stress and the physical and psychological effects of 
stress. This measure is appropriate for adults and adolescents. 

PURPOSE 

Information regarding participants* knowledge of the causes and effects of stress 
maybe useful for the following reasons: 

® Administration of this measure at the beginning of the program 
may provide needs assessment information. For example, the 
results may be used to assess what participants know prior to 
program participation. Decisions about how to allocate 
instructional time can then be made based on the prior 
knowledge of participants. 

• When the measure is administered prior to and following a 
program, it is possible to evaluate growth in participants' 
knowledge. 

PROCEDURES 

^ Because the equidifficulty of the forms has not been established, it is best not to 
give all participants Form A as a pretest and Form B as a posttest. Instead, choose 
either ot the followhg methods. 

• Review Forms A and B and select one. Give all participants the 
selected form both before and after the program. Alternatively, 
select 20 itenis from the two forms and construct a measure most 
consistent with your program emphasis. Then admim'ster the 
"new" form both before and after the program. 

• Give Form A to half of the incoming participants and Form B to 
the remaining half. To distribute the forms randomly, order 
them "ABaBAB" and hand them out. Following the program, 
give each participant the form not previously taken. For 
example, if a participant was given Form B before the program, 
then that participant should be given Form A following the 
program. This approach eliminates the possibiliW that 
examinees will be sensitized to the specific facts to be learned 
from the program. 



ERLC 



SCORING AND ANALYSIS 



The answer keys for the two forms are provided below: 



Item No. 


Form A 


FormB 


1 


T 


F 


2 


T 


F 


3 


F 


T 


4 


F 


F 


5 


T 


F 


6 


F 


T 


7 


T 


T 


8 


F 


T 


9 


F 


T 


10 


T 


F 


11 


F 


F 


12 


T 


F 


13 


F 


T 


14 


T 


T 


15 


F 


F 


16 


T 


F 


17 


T 


T 


18 


T 


T 


19 


F 


T 


20 


F 


F 



The measures should be scored by counting the number of answers correct for 
each participant. Items marked "Don't Know" or left blank should be scored as 
incorre^ Count the number of correct answers for each participant. Next, total the 
correcl vers for the group and divide by the number of participants in the group. 
TTie mta.^ number of correct answers and the standard deviation can be used to 
sunmiarize participant performance on the measure. Means and standard deviations 
from before and after the program can be compared to determine changes in 
participants' knowledge. 
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FACTS ABOUT STRESS 
Form A 



This test consists of 20 statements about stress. Put a check 
to show whether you think each statement is TRUE or 
FALSE. If you don't know whether a statement is true or 
false, put a check under DONT KNOW* 



True 



False DonH Know 



() 



() 



1. Loud noise can be stressful. 

2. Seeing problems as worse than they are can cause 
stress. 

3. Going through many significant life changes at the 
same time rarely increases people's chances of 
getting sick. 

4. A person who is experiencing stress is usually 
sensitive to other people's feelings. 

5. Stress can cause ulcers. 

6. Slow breathing is one of the physical responses to 
stressors. 

7. Stress may decrease the body's ability to defend itself 
against disease. 

8. Being bored is usually not stressful. 

9. A person with a Type A personality typically tries co 
do one thing at a time. 

10. Stress is usually harmful if it lasts for long time. 

n. A person who is experiencing extreme stress thinks 
more clearly than usual. 

( ) 12. Drinking a lot of coffee makes some people anxious. 
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Facts About Stress (Form A), p. 2 
True False Don't Know 

( ) ( ) ( ) 13- Unpleasant thoughts by themselves usually are not 

enough to cause stress. 

( ) ( ) ( ) 14. People may resume bad habits when they are 

experiencing stress. 

( ) ( ) ( ) 15. Almost everyone interprets demanding events in the 

same way. 

( ) ( ) ( ) 16. High blood pressure can damage the heart. 

( ) ( ) ( ) 17. Aperson'sheredity may influence the type of 

stress-related illness that the person gets. 

() () () 18. Experiencing stress may make a person eat more 

than usual. 

( ) ( ) ( ) 19. People's attitudes rarely influence their reactions to 

stressors. 

( ) ( ) ( ) 20. The more someone knows about a stressor, the more 

stress it will cause. 
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FACTS ABOUT STRESS 
FormB 



This test consists of 20 statements about stress* Put a check 
to show whether you think each statement is TRUE or 
FALSE. If you don't kjiow whether a statement is true or 
false, put a check under DONT KNOW. 



True False Don't Know 

( ) ( ) ( ) 1- Too much stimulation is always more stressful than 

too little. 

( ) ( ) ( ) 2. One way the body responds to stressors is to reduce 

blood pressure. 

() () () 3. Stress can result from the demands of any situation. 

( ) ( ) ( ) 4. Stress rarely causes muscular problems. 

( ) ( ) ( ) 5. Thinking about an unpleasant event is never as 

stressful as actually experiencing the event. 

( ) ( ) ( ) 6. Thinkingof oneself as useless can increase one's 

stress. 

( ) ( ) ( ) 7. Getting injured can be stressful. 

( ) ( ) ( ) 8. Stress can improve a person's performance. 

( ) ( ) ( ) 9. People are more likely to have accidents when they 

experience severe stress. 

( ) ( ) ( ) 10. Hormones present when a person experiences stress 

for a long time increase the body's abih'ty to fight 
infection. 

( ) ( ) ( ) 11- An ordinary event that happens every day is not one 

of the usual causes of stress. 

( ) ( ) ( ) 12. A person who is experiencing extreme stress 

performs most tasks better than usual. 
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Facts About Stress (Form B), p. 2 

True False Don't Know. 

( ) ( ) ( ) 13. 

( ) ( ) () 14. 

( ) ( ) ( ) 15. 

( ) ( ) ( ) 16. 

( ) ( ) ( ) 17. 

( ) ( ) ( ) 18. 

( ) ( ) ( ) 19. 

( ) ( ) ( ) 20. 



Some stress is inevitable in day-to-day living. 

Something that causes stress is called a stressor. 

Positive life changes are not stressful. 

Predictable and unpredictable events are equally 
stressful. 

The body's responses to stressors occur without 
thinking. 

The Type A personality is associated with heart 
disease. 

Experiendng stress may make a person drink more 
alcohol than usual. 

Stress is not usually associated with a serious illness 
such as cancer. 
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LEARNING ABOUT STRESS 
(FORMS A &B) 



This knowledge measure examines what participants' know about the ^pes of 
activities and events that cause stress and the physical and psychological effects of 
stress. This measure is appropriate for adolescents and preadolescents. 

PURPOSE 

Information regarding participants' knowledge of the causes and effects of stress 
may be useful for the following reasons: 

• Administration of this measure at the beginning of the program 
may provide needs assessment information. For example, the 
results may be used to assess what participants know prior to 
program participation. Decisions about how to allocate 
instructional time can then be made based on the prior 
knowledge of participants. 

© When the measure is administered prior to and following a 

Erogram, it possible to evaluate growth in participants' 
nowledge. 

PROCEDURES 

^ Because the equidifficulty of the forms has not been established, it is best not to 
give all participants Form A as a pretest and Form B as a posttest. Instead, choose 
either oi the following methods. 

• Review Forms A and B and select one. Give all participants the 
selected form both before and after the program. Alternatively, 
select 15 items from the two forms and construct a measure most 
consistent with your program emphasis. Then administer the 
"new" form both before and after the program. 

• Give Form A to half of the incoming participants and Form B to 
the remaining half. To distribute the fonns randomly, order 
them "ABABAB" and hand them out. Following the program, 
give each participant the form not previously taken. For 
example, if a participant was given Form B before the program, 
then that participant should be given Form A following the 
program. This approach eliminates the possibility that 
examinees will be sensitized to the specific facts to be learned 
from the program. 
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SCORING AND ANALYSIS 
The answer keys for the two forms are provided below: 



Item No. Form A Form B 

1 T T 

2 T F 

3 T F 

4 F T 

5 F T 

6 T F 

7 T F 

8 F F 

9 F T 

10 T T 

11 T T 

12 F T 

13 F F 

14 F F 

15 T F 



The measures should be scored by counting the number of answers correct for 
ezch participant. Items marked "Don't Know" or left blank should be scored as 
incorrect. Count the number of correct answers for each participant. Next, total the 
correct answers for the group and divide by the number of participants in the group. 
The mean number of correct answers and the standard deviation can be used to 
summarize participant performance on the measure. Means and standard deviations 
from before and after the program can be compared to determine changes in 
participant knowledge. 
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LEARNING ABOUT STRESS 
Form A 



This test contains 15 statements about stress. Put a check to show whether 
you think each statement is TRUE or FALSE. If you don't know whether a 
statement is true or false, put a check under DON'T KNOW. 



True False Don't Know 

( ) ( ) ( ) 1. Changing schools can cause stress. 

( ) ( ) ( ) 2. Stress can be harmful when it lasts too long. 

( ) ( ) ( ) 3. Having too much to do at home can cause stress. 

( ) ( ) ( ) 4. People who are feeling a lot of stress do most things better 

than usual. 

() () () 5. A person who is feeling stress usually notices other 

people's feelings. 

( ) ( ) ( ) 6. Unexpected things can cause stress. 

( ) ( ) ( ) 7. Stress can cause muscle problems. 

( ) ( ) ( ) 8. Thinking about something unpleasant never causes as 

much stress as actually doing something unpleasant. 

( ) ( ) ( ) 9. High blood pressure is not that dangerous to your health. 

( ) ( ) ( ) 10. Feeling stress may make a person drink too much alcohol. 

( ) ( ) ( ) 11. The amount of stress people feel depends somewhat on 

how they feel about their surroundings. 

( ) ( ) ( ) 12. Good life changes do not cause stress. 

() () () 13. Ifpeople are careful they can avoid all stress. 

( ) ( ) ( ) 14. The heart slows down in times of stress. 

( ) ( ) ( ) 15. People may feel stress when they can't do what they want. 
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LEARNING ABOUT STRESS 
FormB 



This test contains 15 statements about stress. Put a check to show whether 
you think each statement is TRUE or FALSE. If you don't know whether a 
statement is true or false, put a check under DON'T KNOW. 



True 



False Don't Know 



() 
() 
() 

() 



) 



() 

() 
() 
() 
() 

() 

() 
() 

() 

() 



1. The body uses more vitamins than usual in times of stress. 

2. People usually do not feel stress when they are bored. 

3. The ordinary things that happen every day usually do not 
cause stress. 

4. People may feel stress when they think they are not liked 
by others. 

5. Important changes, like moving to a new city, can cause 
stress. 

6. Just thinking about an unhappy time usually does not cause 
stress. 

7. Only adults feel stress. 

8. Any amount of stress is bad for you. 

9. Feeling stress may make a person eat less than usual. 

10. People may feel stress when they want to control 
something and can^t. 

1 1. People may feel stress when they have too many things to 
do, 

12. A person who is feeling stress may seem happy. 

13. Having too much to do always causes more stress than 
having too little to do. 

14. People who are feeling a lot of stress think more clearly 
than usual. 

15. Breathing slows down during times of stress. 
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COPING WITH STRESS 
(FORMS A &B) 



This knowledge measure examines what participants' know about the techniques 
that aid in the management and/or reduction of stress. This measure is appropriate 
for adults and adolescents. 

PURPOSE 

Information regarding participants' knowledge of the causes and effects of stress 
may be useful for the following reasons: 

• Administration of this measure at the beginning of the program 
may provide needs assessment information. For example, the 
results may be used to assess what participants know prior to 
program participation. Decisions about how to allocate 
mstructional time can then be made based on the prior 
knowledge of participants. 

© When the measure is administered prior to and following a 
program, it is possible to evaluate growth in participants' 
knowledge. 

PROCEDURES 

^ Because the equidifficulty of the forms has not been established, it is best not to 
give all participants Form A as a pretest and Form B as a posttest. Instead, choose 
either or the following methods. 

9 Review Forms A and B and select one. Give all participants the 
selected form both before and after the program. Alternatively, 
select 20 items from the two forms and construct a measure most 
consistent with your program emphasis. Then admim'ster the 
"new" form both before and after the program. 

• Give Form A to half of the incoming participants and Form B to 
the remaining half. To distribute the forms randomly, order 
them "ABABAB" and hand them out. Following the program, 
give each participant the form not previously taken. For 
example, if a participant was given Form B before the program, 
then that participant should be given Form A following the 
program. This approach elimmates the possibility that 
examinees will be sensitized to the specific facts to be learned 
from the program. 
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SCORING AND ANALYSIS 



The answer keys for the two forms are provided below: 



Item No. 


Form A 


Form B 


1 


T 


F 


2 


F 


F 


3 


F 


T 


4 


T 


F 


5 


T 


T 


6 


F 


T 


7 


T 


F 


8 


F 


F 


9 


T 


T 


10 


T 


T 


11 


T 


F 


12 


F 


T 


13 


T 


T 


14 


F 


T 


15 


F 


F 


16 


F 


F 


17 


T 


T 


18 


F 


F 


19 


F 


T 


20 


T 


F 



The measures should be scored by counting the number of answers correct for 
each participant. Items marked "Don't Know'' or left blank should be scored as 
incorrect. Count the number of correct answers for each participant. Next, total the 
correct answers for the group and divide by the number of participants in the group. 
The mean number of correct answers and the standard deviation can be used to 
summarize participant performance on the measure. Means and standard deviations 
from before and after the program can be compared to determine changes in 
participant knowledge. 
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COPING WITH STRESS 
Form A 



This test consists of 20 statements about stress. Put a 
check to show whether you think each statement is TRUE 
or FALSE. If you don't know whether a statement is true 
or false, put a check under DON'T KNOW. 



True 



False Don't Know 



() 
() 

() 

() 

() 

() 

() 

() 
() 

() 

() 



1. Having good friends helps to reduce stress. 

2. The human body needs more caffeine when a 
person is anxious. 

3. Leaving the most important task for last helps 
to reduce stress. 

4. Relaxation techniques help people become 
more aware of how their bodies respond to 
stress. 

5. Setting up a daily schedule often helps to 
reduce stress. 

6. It is better to put up with a stressful situation 
than to try to change it. 

7. In many forms of meditation, attention is 
focused on one thing, such as a word or sound. 

8. People can avoid all stressors if they want to. 

9. Walking can be an effective way to reduce 
stress. 

10. Relaxation relieves many muscular problems 
caused by stress. 

11. Planning activities for periods of boredom 
often helps to reduce stress. 
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Coping with Stress (Form A), p. 2 

True False Don't Know. 

( ) ( ) ( ) 12. 

( ) ( ) ( ) 13. 

( ) ( ) ( ) 14. 

( ) ( ) ( ) 15. 

( ) ( ) ( ) 16. 

( ) ( ) ( ) 17. 

( ) ( ) ( ) 18. 

( ) ( ) ( ) 19. 

( ) ( ) ( ) 20. 



When exercising to reduce stress, people 
should try to push themselves as much as 
possible. 

One way to reduce stress is to become less 
competitive. 

When a person feels overwhelmed, delegating 
responsibility to others will increase stress. 

To be effective, relaxation techniques must be 
practiced at the same time each day. 

People usually feel more stress if they talk to 
others who have similar troubles. 

Biofeedback uses an instrument to monitor 
certain changes in the body. 

One characteristic of relaxation is faster 
breathing. 

Regular physical activitj' will increase the 
effects of stressors. 

Finding alternatives for goals that one has 
been unable to accomplish often helps to 
reduce stress. 
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COPING \WrH STRESS 
FormB 



This test consists of 20 statements about stress. Put a 
check to show whether you think each statement is TRUE 
or FALSE, If you don^t know whether a statement is true 
or false, put a check under DON'T KNOW* 



True FaJse Don't Know 

( ) ( ) ( ) 1. Talking to others who have similar troubles 

rarely reduces stress. 

( ) ( ) ( ) 2. When starting up an exercise program to 

reduce stress, one should use vigorous 
activities. 

( ) ( ) ( ) 3. Breaking down complicated tasks into smaller 

parts often reduces stress. 

( ) ( ) ( ) 4. People should avoid all situations in which 

they experience stress. 

( ) ( ) ( ) 5. One way to reduce stress is to avoid changes 

that are not necessary. 

( ) ( ) ( ) 6. A person's heartbeat can be mom'tored by 

biofeedback. 

( ) ( ) ( ) 7. It is not vital to identify situations that cause 

stress. 

( ) ( ) ( ) 8. When trying to reduce stress, people should 

change their behavior rather than the 
environment. 

( ) ( ) ( ) 9. One way to reduce stress is to make a plan of 

action and follow through with it. 

( ) ( ) ( ) 10. Getting one's mind off a problem by doing 

something else for awhile often reduces stress. 
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Coping with Stress (Form B), p, 2 



True False Don't Know 

() () () IL Everyone should cope with similar stressors in 

the same way, 

( ) ( ) ( ) 12, Progressive relaxation helps to relieve 

backaches caused by stress, 

( ) ( ) ( ) 13, Getting information about a stressor often 

helps to reduce stress, 

( ) ( ) ( ) 14, People who can effectively manage their time 

are less likely to feel stress, 

( ) ( ) ( ) 15, Hoping people vwll be perfect is an effective 

way to reduce stress, 

( ) ( ) ( ) 16, Meditating usually does not help people relax, 

( ) ( ) ( ) 17. Focusing on one's positive qualities often 

helps to reduce stress, 

( ) ( ) ( ) 18, A faster heart rate is one characteristic of 

relaxation, 

( ) ( ) ( ) 19, Thinking through how one imght handle a 

stressor often reduces stress, 

( ) ( ) ( ) 20, To be effective, relaxation techniques must be 

done in the same place each day. 
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WAYS TO LOWER STRESS 
(FORMS A &B) 



This knowledge measure examines what participants' know about the techniques 
that aid in the management and/or reduction of stress. This measure is appropriate 
for adolescents and pre adolescents. 

PURPOSE 

Information regarding participants' knowledge of the causes and effects of stress 
may be useful for the following reasons: 

• A.dministration of this measure at the beginning of the program 
may provide needs assessment information. For example, the 
results may be used to assess what participants know prior to 
program participation. Decisions about how to allocate 
instructional time can then be made based on the prior 
knowledge of participants, 

© When the measure is administered prior to and following a 
program, it is possible to evaluate growth in participants' 
knowledge. 

PROCEDURES 

^ Because the equidifficulty of the forms has not been established, it is best not to 
give all participants Form A as a pretest and Form B as a posttest. Instead, choose 
either or the following methods. 

• Review Forms A and B and select one. Give all participants the 
selected form both before and after the program. Alternatively, 
select 15 items from the two forms and construct a measure most 
consistent with your program emphasis. Then administer the 
"new" form both before and after the program. 

o Give Form A to half of the incoming participants and Form B to 
the remaining half. To distribute the forms randomly, order 
them "ABABAB" and hand them out. Following the program, 
give each participant the form not previously taken. For 
example, if a participant was given Form B before the program, 
then that participant should be given Form A following the 
program. This approach elimmates the possibility that 
examinees will be sensitized to the specific facts to be learned 
from the program. 
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SCORING AND ANALYSIS 

The answer keys for the two forms are provided below: 



Item No. Form A Form B 

1 T T 

2 F T 

3 F T 

4 T T 

5 T T 

6 F F 

7 T F 

8 T T 

9 T F 

10 T F 

11 F T 

12 F F 

13 F T 

14 T F 

15 F F 



The measures should be scored by counting the number of answers correct for 
each participant. Items marked "Don't Know" or left blank should be scored as 
incorrect. Count the number of correct answers for each participant. Next, total the 
correct answers for the group and divide by the number of participants in the group. 
The mean number of correct answers and the standard deviation can be used to 
summarize participant performance on the measure. Means and standard deviations 
from before and after the program can be compared to determine changes in 
participant knowledge. 
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WAYS TO LOWER STRESS 
Form A 



This test consists of 15 statements about how to reduce 
stress. Put a check to show whether you think each statement 
is TRUE or FALSE, If you don't know whether a statement is 
true or false, put a check under DONT KNOW. 



True 



False Don't ICnow 



() 



() 



() 



ERIC 



1. People should try to make plans if they are bored. 

2. People usually feel more stress if they talk to others 
who have similar problems. 

3. r eople who have too many thiugs to do should save 
the most important things for last. 

4. Eating a balanced diet is important when a person is 
feeling stress. 

5. One way to lower stress is to avoid unnecessary 
changes. 

6. Exercise helps people lower their stress if they push 
themselves as hard as they can. 

7. A person should try to put up with a situation if 
nothing can be done to change it. 

8. Many people can lower their stress by making a plan 
of action and following it. 

9. Many people can lower their stress by talking about 
their feelings with friends. 

10. Jogging helps many people lower their stress. 

11. People can avoid all stressful situations if they want to. 

12. A person meditates by thinking about several things at 
once. 

13. People can force themselves to relax. 

14. Thinking about the good things in life often helps to 
lower stress. 

15. A person breathes faster during relaxation. 
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WAYS TO LOWER STRESS 
FormB 



This test consists of 15 statements about how to reduce 
stress. Put a check to show whether you think each statement 
is TRUE or FALSE. If you don't know whether a statement is 
true or false, put a check under DON'T KNOW. 



True 
() 



False Don't Know 

( ) ( ) 1. Many people can lower their stress by having good 

friends. 

2. Taking deep breaths helps a person relax. 

3. People with too much to do can lower their stress by 
breaking down long jobs into small parts, 

4. People can learn ways to lower their stress. 

5. Exercise helps many people lower their stress. 

6. The heart beats faster when a person is relaxed. 

7. Expecting everyone to be perfect helps to lower a 
person's stress. 

8. Relaxation is the opposite of what normally happens 
to people under stress. 

9. People should have more caffeine when they are 
nervous. 

10. Making day-to-day personal schedules of activities 
increases stress. 

11. It is helpful for people to learn which situations cause 
them to feel stress. 

12. To lower their stress, people should work on difficult 
jobs without taking any breaks. 

13. Many people can lower their stress by thinking 
through how to handle problems. 

14. People who get more information about something 
that worries them usually feel more stress. 

15. All people should lower their stress in the same way. 
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APPROPRIATE RESPONSES TO STRESS 
(FORMS A &B) 



This skill measure assesses participants' ability to respond appropriately to 
stressful situations* This measure is appropriate for adults. 

PURPOSE 

Information regarding participants' ability to respond appropriately to stressful 
situations may be useful for the following reasons: 

• Administration of this measure at the beginning of the program 
may provide needs assessment information. For example, the 
results may be used to assess how participants respond prior to 
program participation. Decisions about how to allocate 
instructional time can then be made based on the prior skills of 
parddpants. 

^ When the measure is administered prior to and following a 
program, it is possible to evaluate growth in participants' skill 
level. 

PROCEDURES 

Because the equidifficulty of the forms has not been established, it is best not to 
give all participants Form A as a pretest and Form B as a posttest. Instead, choose 
either or the following methods. 

• Review Forms A and B and select one. Give all participants the 
selected form both before and after the program. Alternatively, 
select 15 items from the two forms and construct a measure most 
consistent with your program emphasis. Then admim'ster the 
"new" form both before and after the program. 

• Give Form A to half of the incoming participants and Form B to 
the remaining half. To distribute the forms randomly, order 
them "ABABAB" and hand them out. Following the program, 
give each participant the form not previously taken. For 
example, if a participant was given Form B before the program, 
then that participant should be given Form A following the 
program. This approach eliminates the possibility that 
examinees will remember how they answered each item on the 
pretest. 
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SCORING AND ANALYSIS 



The correct answer keys for the two forms are provided below: 



Item No. 


Form A 


FormB 


1 


D 


C 


2 


B 


D 


3 


D 


C 


4 


B 


B 


5 


A 


C 


6 


C 


D 


7 


C 


C 


8 


D 


B 


9 


B 


B 


10 


A 


C 


11 


C 


D 


12 


D 


B 


13 


B 


D 


14 


A 


B 


15 


A 


C 



The measures should be scored by counting the number of correct answers foi 
each participant. Blank items should be scored as incorrect. Count the number of 
correct answers across participants. The mean number of correct answers and the 
standard deviation can be calculated to summarize participant performance on the 
measure. Means and standard deviations from before and after the program can be 
compared to determine changes in participants' skill. 
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APPROPPJATE RESPONSES TO STRESS 
Form A 



This test presents descriptions of individuals who are in 
stressful situations. These people want to reduce their stress. 
Read each item. Circle the letter of the appropriate action for 
the person to take to reduce the stress. If there is no choice 
presented that is appropriate, circle choice D, "None of the 
above. 



Last week Ray got divorced after being married for ten years. An appropriate way 
for Ray to reduce his stress would be to: 

A. Move to a new city to start a new life. 

B. Spend more time drinking at his neighborhood bar. 

C Stop seeing any friends that he and his wife had in common. 
D. None of the above. 

Martha has many chores that must be fmshed by the end of the day. Martha's friend, 
Cheryl, calls up Martha to ask for help painting her living room that day. Martha 
feels that she can't help Cheryl and finish her own work as well. An appropriate way 
for Martha to reduce her stress would be to: 

A. Grudgingly help Cheryl paint but try to get it done as quickly as possible. 

B. Explain to Cheryl that she can't help ter and continue working on her own 
chores. 

C Invite Cheryl to go out shopping with her so they both can get their minds off 
their work. 

D. None of the above. 



Appropriate Rer^onses to Stress (Form A), p. 2 

3. Dave is flying out of state for a business trip and is terrified of flying in an airplane. 
An appropriate way for Dave to reduce his stress would be to: 

A. Drink a few glasses of wine before the flight. 

B. Think about how fast the plane is flying. 

C. Get to the airport late so that he doesn't have as much time to worry. 

D. None of the above. 

4. Steve is very busy typing when a co-v/orker asks Steve to help her with her typing. 
Steve is a bit annoyed by her request. An appropriate way for Steve to reduce his 
stress would be to: 

A. Help her with her typing but honestly express annoyance and ask that she not 
request help again. 

B. Explai J that he is too busy to do her typing and finish his own work. 

C. Help her with her typing, but tell her that he doesn't think the boss would 
approve. 

D. None of the above. 

5. Roberto was called into his supervisors office and told that the company must lay 
off some employees. Roberto will be laid off in two months. An appropriate way for 
Roberto to reduce his stress would be to: 

A. Get some informafion about different job opfions. 

B. Avoid the other employees who are being laid off. 

C. Show the supervisor how he feels by quitting on the spot. 

D. None of the above. 

6. Liz is going through a busy time at work. She has several projects due at the same 
fime. Liz is anxious because she doesn't think she will be able to complete 
everything on fime. An appropriate way for Liz to reduce her stress would be to: 

A. Plan to avoid breaks and to work through lunch hours in order to finish the 
projects. 

B. Work rapidly without worrying about any of the individual deadlines, but 
concentrate on getfing everything finished at the same time. 

C. Set up a schedule and work on the most important project first. 

D. None of the above. 
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Appropriate Responses to Stress (Form A), p. 3 

7. Gary is concerned that the quality of his work is not good enough, even though all of 
the people he works with tell him he's doing a good job. An appropriate way for 
Gary to reduce his stress would be to: 

A* Make further efforts to improve the quality of his wt ^rk. 

B. Drink a few beers at lunch to help him relax at work. 

C. Focus on the positive qualities of his work. 

D. None of the above. 

8. Stan sits in traffic for an hour every day on the way to work. An appropriate way for 
Stan to reduce his stress would be to: 

A. Bring a container of coffee to drink in the car. 

B. Turn up the volume on the radio in his car. 

C Complain about the traffic to the people at work. 
None of the above. 

9. Shelly works in the ticket booth at a movie theatre. She has to stay in the booth while 
the movie is being shown, even if there are no people waiting to buy tickets. An 
appropriate way for Shelley to reduce her stress would be to: 

A. Take some food to nibble on during work. 

B. Ask her boss if she can read a book when no one is in line* 

C. Bring a soft pillow for her chair. 

D. None of the above. 

10. Dave is planning to go on a rafting trip next month, and has just been told that his 
trip has been cancelled. The trip was arranged by a travel agent who was 
recommended by a friend. An appropriate way for Dave to reduce his stress would 
be to: 

A. Plan another vacation for the same time period. 

B. Spend his vacation at home and find a book about river rafting. 

C. Complain to his friend about his recommendation. 

D. None of the above. 
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Appropriate Responses to Stress (Form A), p. 4 

11. Maria has four final exams and only two days left to study for them. An appropriate 
way for Maria to reduce her stress would be to: 

A. Take her mind off her own tests by helping a friend study. 

B. Pick the hardest course and spend most of her time studyin ; for that exam. 

C. Set up a schedule so that she has some time to study for each test. 

D. None of the above. 

12. Greg lives across from an all mght gas station and is disturbed by the noise from the 
cars. An appropriate way for Greg to reduce his stress would be to: 

A. Turn up his stereo to block out the noise. 

B. Take a sleeping pill to help get to sleep. 

C. Stay up later so that he can fall asleep more easily. 

D. None of the above. 

13. Tom recently bought a new business and is working seven days a week to keep it 
going. An appropriate way for Tom to reduce his stress would be: 

A. Close the business for several days and go on a vacation. 

B. Set aside some time each week to exercise. 

C. Skip lunch so he can close the business earlier each day. 

D. None of the above. 

14. Melanie and her son Mark have just had an argument about the trouble Mark is 
getting into at school. An appropriate way for Melam'e to reduce her stress would be 
to: 

A. Talk about the situation with one of the counselors at school. 

B. Have a counselor tell Mark he will be suspended, even if it's not true. 

C. Think about what she should have done in the past to help Mark with school. 

D. None of the above. 
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Appropriate Responses to Stress (Form A), p. 5 

15, Joyce must speak to a large group of people. She is well-prepared to give the speech. 
However, she keeps remembering another time when she gave a speech and forgot 
what she was supposed to say. An appropriate way for Joyce to reduce her stress 
would be to: 

a\ Set aside some time to sit quietly before the speech. 

B. Look directly at the audience while she gives her speech. 

C. Keep her hands busy while she gives the speech. 

D. None of the above. 



APPROPRIATE RESPONSES TO STRESS 
FormB 



This test presents descriptions of individuals who are in 
stressful situations. These people want to reduce their stress. 
Read each item. Circle the letter of the appropriate action for 
the person to take to reduce the stress. If there is no choice 
presented that is appropriate, circle choice ^^None of the 
above." 



1. Valerie has just taken another job in a different city. She's feeling overwhelmed with 
all the new things she's encountering. An appropriate way for Valerie to reduce her 
stress would be to: 

A. Change her hairstyle and way of dressing to reflect her new image. 

B. Take on a lot of projects at work to keep herself busy. 

C. Establish a suitable daily schedule soon after she arrives. 

D. None of the above. 

2. John is in a noisy office and is trying to concentrate on his work. An appropriate way 
for John to reduce his stress would be to: 

A. Skip lunch and work during lunch when the office is quieter. 

B. Straighten up his office area. 

C. Wear more comfortable clothes to work. 

D. None of the above. 
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Appropriate Responses to Stress (Form B), p. 2 



3. Barbara's husband has started a new job and is taking many out-of-town business 
trips. Barbara is bored because she has more time on her hands than she's used to 
having. An appropriate way for Barbara to reduce her stress would be to: 

A. Ask her husband to look for another job. 

B. Talk to her husband's bosi about reducing the frequent trips. 

C. Get involved in an activity to fill the time. 

D. None of the above. 

4. Bill has just found out that his mother is seriously ilL He is very worried about her. 
An appropriate way for Bill to reduce his stress would be to: 

A. Tell the doctor to spend more time with his mother. 

B. Talk to his family about his concerns about his mother's health. 

C. Take a tranquilizer whenever he feels himself getting very upset. 

D. None of the above. 

5. Rod is worried because he and his family are in financial trouble* He is unable to 
pay all of the monthly bills and is slipping further into debt. An appropriate way for 
Rod to reduce his stress would be to: 

A. Pay those bills that he can afford and forget about the rest for awhile.' 

B. Not mention the situation to his family until it's absolutely necessary. 

C. Set up payment schedules with the different bill collectors. 

D. None of the above. 

6. Nancy drives home on a busy, crowded freeway. An appropriate way for Nancy to 
reduce her stress would be to: 

A. Think about all the things she could be doing with the time she spends on the 
freeway. 

B. Try to take the same route home whenever possible. 

C. Eat something while she is driving to keep her mind off the traffic. 

D. None of the above. 
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Appropriate Responses to Stress (Form B), p. 3 

7. Jeff missed out on a promotion at his office because he didn't have the skills for the 
higher position. An appropriate way for Jeff to reduce his stress would be to: 

A* Continue working in his current position but with less effort. 
Be friendly to the person who did get the job- 

C. Talk to his boss about ways he can improve his skills. 

D. None of the above. 

8. Phil is feeling nervous because he and his wife have just recently moved to a new 
city. An appropriate way for Phil to reduce his stress would be to: 

A. Spend some time drinking in a neighborhood bar. 

B. Buy some guide books to learn about the city. 

C. Go out of town on a vacation with his wife. 

D. None of the above. 

9. Sharon works on an assembly line where she watches metal fittings go by all day 
long. An appropriate way for Sharon to reduce her stress would be to: 

A, Put up a poster near where she vorks on the assembly line. 

B, Ask her boss if she can listen to a radio as she works. 

C. See if she can work through lunch so that she can finish her work as quickly as 
possible. 

D. None of the above. 

10. Carmela had been planning on taking a week off from work. Now her boss tells 
Carmela that it is impossible for her to have the vacation time she had planned. An 
appropriate way for Carmela to reduce her stress would be to: 

A. Say she will switch jobs unless she can take her vacation as planned. 

B. Use the opportunity to ask for a raise because she didn't get her vacation time. 

C. Tell her boss that she's disappointed and make plans for her rescheduled 
vacation. 

D. None of the above. 
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Appropriate Responses to Stress (Form B), p. 4 

11. Rosalie is moving to a new apartment next weekend. She has to pack all hei things 
and c'ean her old apartment in two days. An appropriate way for Rosalie to reduce 
her stress would be to: 

A. Maintain her normal routine, and stay up late to do her packing. 

B. Drink coffee so she has the energy she needs for her move. 

C. Avoid all her friends until she is settled in her new place. 

D. None of the above. 

12. Gwen is upset because she has had several arguments with her mother. Gwen wants 
her mother to move into a nursing home. Her mother refuses to live in a nursing 
home and wants to continue living alone in an apartment. An appropriate way for 
Gwen to reduce her stress would be to: 

A. Continue trying to convince her mother to live in a nursing home. 

B. Talk to her mother about other living arrangements that might be suitable. 

C. Show her mother how upset she is by not contacting her for a few weeks. 

D. None of the above. 

13. Ginger has a very long paper to write for a class she is taking. She is nervous because 
she can't seem to get started on the paper. An appropriate way for Ginger to reduce 
her stress would be to: 

A. Think about a way to get her teacher to postpone the due date. 

B. Not show up for class the day the paper is due. 

C. Tliink about how much work is involved in writing the paper. 

D. None of the above. 

14. Peg's doctoi told her that she couldn't play in aa upcoming tennis tournament 
because she pulled a muscle in her leg. An appropriate way for Peg to reduce her 
stress would be to: 

A. Wrap her leg in a bandage and play in the tournament anyway. 

B. Stay home from the tournament and do something she enjoys. 

C. Go out drinking with her friends to have a good time. 

D. None of the above. 
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Appropriate Responses to Stress (Form B), p. 5 

15. Neil is worried that he won't do well answering questions at his job interview, even 
though he is qualified for the job. An appropriate way for Neil to reduce his stress 
would be to: 

A. Think about how upset his wife will be if he doesn't get the job. 

B. Stay up late the night before the interview to prepare something to say. 

C. Take one of his regular morning jogs before the interview. 

D. None of the above. 



S3 



HOW TO LOWER STRESS 
(FORM A A B) 



This skill measure assesses participants' ability to respond appropriately to 
stressful situations. This measure is appropriate ^or adolescents and preadolescents. 

PURPOSE 

^ Information regarding participants' ability to respond appropriately to stressful 
situations may be useful for the following reasons: 

• Administration of this measure at the beginning of the program 
may provide needs assessment information. For example, the 
results may be used to assess how participants respond prior to 
program participation. Decisions about how to allocate 
mstructional time can then be made based on the prior skills of 
participants. 

• When the measure is administered prior to and following a 
program, it is possible to evaluate growth in participants' skill 
level. 

PROCEDURES 

^ Because the equidifficulty of the forms has not been established, it is best not to 
give all participants Form A as a pretest and Form B as a posttest. Instead, choose 
either of the following methods. 

e Review Forms A and B and select one. Give all participants the 
selected form both before and after the program. Alternatively, 
select 10 items from the two forms and construct a measure most 
consistent with your program emphasis. Then administer the 
"new" form both before and after the program. 

• Give Form A to half of the incoming participants and Form B to 
the remaimng half. To distribrte the forms randomly, order 
them "ABABAB" and hand them out. Following the program, 
give each participant the form not previou y taken. For 
example, if a participant was given Form B befoie the program, 
then that participant should be given Form A following the 
program. This approach elimmates the possibility that 
examinees will remember how they answered each item from 
the pretest. 
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SCORING AND ANALYSIS 

The correct answer keys for the two forms are provided below: 



Item No. 


Form A 


FormB 


1 


C 


B 


2 


B 


A 


3 


B 


B 


4 


A 


c 


5 


C 


B 


6 


A 


B 


7 


A 


C 


8 


B 


B 


9 


E 


B 


10 


C 


C 



The measures should be scored by counting the number of correct answers for 
each participant. Blank items should be scored as incorrect. Count the number of 
correct answers across participants. The mean number of correct answers and the 
standard deviation can be calculated to summarize participant performance on the 
measure. Means and standard deviations from before and after the progiam can be 
compared to determine changes in participants' skill. 
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HOW TO LOWER STRESS 
Form A 



This test is about young people who feel stress or arc nervous 
because of whaJ is happening to them. These people want to 
lower their stress. Read each description. Circle the letter of the 
best action for the person to take to lower the stress. 



1. Jan wants to go to a party with her older sisten She knows her father won't let her go 
because most of the people at the party will be a lot older than Jan. To lower her 
stress Jan should: 

A. Go to the party without telling her father. 

B. Ask her sister to tell her father that most of the kids will be Jan's age. 
C Ask her dad if she can go to the movies instead. 

2. Derrick just got his report card, and his grades are not good. He is afraid his parents 
will be upset, even though tliey are usually very understanding. To lower his stress 
Derrick should: 

A. Plan to stay up late during the next grading period to study more. 

B. Think about similar situations in the past in which his parents were 
understanding. 

C Do some jobs around the house to make up for his grades. 

3. Bill is feeling nervous because he is surrounded by people at a very crowded dance. 
To lower his stress Bill should: 

A. Get over to the food table to eat something. 

B. Go outside and take a short walk. 
C Stay in the middle of the crowd. 
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How to Lower Stress (Form A), p. 2 

4. Stacey is completing the sixth grade soon and will be going to a new school for 
seventh grade. To lower her stress Stacey should: 

A. Visit the new school to get familiar with it before classes start. 

B. Think about how much she likes her sixth grade class. 

C Ask her parents if the family can take a vacation together before school starts. 

5. Jo3'ce has practiced a speech and has memorized it welL She still thinks that she will 
forget what she's supposed to say when she gives it to her class. To lower her stress 
Joyce should: 

A. Try not to look at anyone while she gives the speech. 

B. Think about how upset the teacher will be if she forgets her speech. 

C. Tell herself that she knows her speech and will do well. 

6. Jamal is feeling uncomfortable because his mother and father have just had a big 
jBght. To lower his stress Jamal should: 

A. Talk to his older brother about the fight. 

B. Think about whether he was the cause of the fight. 

C. Not do his chores to show how upset he is. 

7. Evelyn is trying to figure out how to run a computer program, but the directions are 
very difficult. To lower her stress Evelyn "hould: 

A. Take a break and see if the directions make more sense when she gets back. 

B. Keep working on the program through the lunch period. 

C. Tell the people around her to be quieter so that she can finish her program. 

8. Sean's school is selling candy bars to earn money. Sean must sell 3 boxes of candy 
bars. He has waited until the last 3 days to start selling them. To lower his stress 
Sean should: 

A. Buy one of the boxes of candy bars to share with his friends. 

B. Make a plan to try to sell one box of candy ba^s each day. 

C. Decide that it is too late to sell any so he will ask his parents to buy them. 
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How to Lower Stress (Form A), p. 3 



9. Brenda enjoys being with her best friend. This year she had hoped to be in the same 
class as her friend, but they were pui in different classrooms. To lower iier stress 
Brenda should: 

A. Use her lunch hour to talk with her friend instead of eating. 

B. Plan some after-s^^iiool activities to do with her friend. 

C. Think about all the fiin times she and her friend have had together. 

10. Kevin was given c* note from his teacher telling him to go to the principal's office 
later that same day. He is afraid he is in trouble, but he knows he hasn't done 
anything wrong. To lower his stress Kevin should: 

A Try to think of something he might have done wrong in the past. 

B. Try to figai^ out what to tell his parents if he is in trouble. 

C. Tell himself that he hasn't done anything wrong and that he has no reason to be 
frightened. 



88 



HOW TO LOWER STRESS 
FormB 



This test is about young people who feel stress or are nervous 
because of what is haj^ pening to them. These people want to 
lower their stress. Read each description. Circle the letter of the 
best action for the person to take to lower the stress. 



1. Darnell is supposed tc- go to the hospital with his parents to visit his grandmother. 
Darnell is afraid of hospitals. Although he would like to see his grandmother, he 
doesn't want to go to the hospital. To lower his stress Darnell should: 

A. Tell his parents that he will go to the hospital the next time they visit, but not 
this time. 

B. Talk to his parents about how frightened he is. 

C. Go with his parents, hoping that his fear will go away at the hospital. 

2. Mark is trying to finish this week's history lesson, but the lesson is veiy long. To 
lower his stress Mark should: 

A. Divide the lesson into smaller parts and work on each part at a different time. 

B. Stay up very late one night to tiy to finish the lesson. 

C. Study for his other classes before starting his history lesson. 

3. Bridgette, who has just finished seventh grade, has nothing to do during sumnier 
vacation and is very bored. To lower her stress Bridgette should: 

A. Stay in the house and take it easy. 

B. Try to make some new friends during the summer. 

C. Wait ^/ound and see if her friends call her. 
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How to Iwower Stress (Form B), p. 2 



4. Lisa is trying out for the school play. She has practiced reading the parts she is 
interested in and knows them well, but she is frightened that she will stumble over 
them. To lower her stress Lisa should: 

P . Plan on trying out for the next play and forget about being in this one. 

B. Get something to eat from a school candy machine. 

C. Sit in a quiet place before she has to i oad her lines. 

5. Lewis just heard his parents yelling at each other. They didn*t know he was in the 
next room. To lower his stress Lewis should: 

A. Stay in the room until he is sure that his parents are gone. 

B. Talk to his parerts individually about what happened. 

C. Not say anything about the fight to his parents to keep them from getting more 
angry. 

6. Melissa is trying to solve her algebra problems but is getting more and more 
confused. To lower her stress Melissa should: 

A. Skip lunch to keep working on the project. 

B. Take a break and try again when she gets back. 

C Read the book over and ove * until she figures them out. 

7- Arthur is worried that he will net pass tomorrow's geography test, even though he 
has studied what he needs to know. To lower his stress Arthur should: 

A. Stay up late to s'oidy more. 

B. Think about what might happen if he fails the test. 

C. Go outside and play basketball with his brother. 

8. Jason has been told that he did not get chosen for the school soccer team. To lower 
his stress Jason should: 

A. Avoid the kids who did make the team. 

B. Ask the coach if there is a city tean that he could join. 

C. Tell the coach that he had been thinking of quitting anyway. 
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How to Lower Stress (Form B), p. 3 



9. Rosa and her family have just moved. Rosa will be startii^g at a new school soon. To 
lower her stress Rosa should: 

A. Stop writing to friends that remind her of her old school. 

B. Visit her new school to learn where things are. 

C. Think about how much she misses her old classmates. 

10. Jody wants to go see the fireworks show on the Fourth of July. Her mother wili not 
let her go out because Jody has just gotten over a bad cold. To lower her stress Jody 
should: 

A. Keep asking her mother if she can go watch the fireworks show. 

B. Stay in her room all evening to show her mom how angry she is. 

C Ask her mother if she may make a special dinner at home that night. 
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HOW WlL . YOU FEEL? 



This affective measure asks participants to indicate the importance of 12 factors 
that influence their quality of life. After weighing the importance of each factor, 
participants are asked to predict how satisfied they expect to be in one year. This 
measure is appropriate for adults. 

If this measure seems useful, you might also want to consider the Life Satisfaction 
Inventory (LSI), which is another affective measure assessing participants' 
perceptions about their quality of life. Th- LSI measure asks about current 
satisfaction levels rather than expected satisfacuon levels. 

PURPOSE 

Information about participants' expected life satisfaction may be useful for the 
following reasons: 

© Admim'stration of this measure ?t the beginning of the 
prograir may provide needs assessment information. For 
example, results of this measure may show that participants 
have low expectations about their future quality of life, thus 
emphasizing their need for instruction in stress management 
techniques. If participants feel that they are better able to 
handle stress, their expected life satisfaction may also increase. 

9 When this measure is administered prior to and following a 
program, it is possible to evaluate changes in participants' 
expectation of their ftitui'e quality of life in vvvelve distinctive 
areas. 



PROCEDURES 

This instrument can be admim'stered both at the beginning and at the end of the 
program. However, handbook users should be alert to concerns regarding the 
potential reactivity of affective measures. A measure is considered reactive if the 
experience of completing the measure prior to the program causes participants to 
react differently to the program. Handbook users should, therefore, carefully review 
each affective measure that they wish to use to determine its potential for making 
participants unduly sensitive to aspects of the program. If a measure is determined to 
be reactive, then program personnel should not administer that measure to all 
participants as a pretest and posttest. Instead, the measure could be administered to 
half Or the pro.'^ram participants prior to program participation to determine 
participants' pre-program status. The measure could then be administered to the 
other half of the participants after i^rogram participation to assess participants' 
post-program status. 



SCORING AND ANALYSIS 

This inventory can be scored in two ways. One procedure relies on the 
EXPECTED SATISFACTION responses only, providing an easily obtained group 
estimate of participants' expected lite satisfaction in one year. 
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A second procedure combines the IMPORTANCE responses with the 
EXPECTED SATISFACnON responses. This method provides a weighted 
assessment of expected life satisfaction based on the IMPORTANCE rating. 

Procedure 1 - Expected Satisfaction Responses Only 

1. Add the point values of all the responses in Column 2 for all participants. 
(Ignore blanks; that is, do not count them when you count the number of 
responses.) 

2. Divide this total by the total number of responses. 

EXAMPLE: Imagine that there are 10 program participants. First, add up all 
the numbers in Column 2 for all participants. Let's say the total is 190. Divide 
this total by the total number of responses from all participants. We will use 
110 responses for this example. (The total possible number of responses is 120 
if all participants left no blanks in Column 2.) Then divide 190 by 110 
responses to get an average score of about 1.7. 

T-ie maximum total EXPECTED SATISFACTION score is 3.0; the minimum 
score is 1.0. Scores near 3.0 indicate that participants as a group expect to be very 
satisfied with their quality of life. Scores near 1.0 indicate that participants anticipate 
being dissatisfied. 

Procedure 2 ~ Expected Satisfaction Weighted by Importance Responses 

1. Count the number of items rated "very important" (3) in Column 1. 

2. Total the EXPECTED SATISFACnON ratings in Column 2 for the items 
counted in step 1. 

3. E .vdde tne total in step 2 by the total in step 1. 

EXAMPLE: For the same 10 participants used in the example above, count 
the number of items that they rated as '"very important" in Column 1. Of the 
120 responses possible, let's say there were 43 responses marked with a 3. For 
those same 43 responses, move over to Column 2 and add up the expected 
satisfaction ratings given to them. Let's assume a total of 116. Then, divide 
116 by 43 for an EXPECTED SATISFACTION average score of 2.7 which is 
weighted by participants' '"very important rating. 

Again, the maximum attainable score is 3.0; the minimum is 1.0. Scores close to 
3.0 indicate that participants expect to be very satisfied with those factors rated "very 
important." 

This same procedure can also be followed to determine participants' expectation 
for those items rated "somewhat important." 
NOTES: 

(1) If both^ this measure and the Life Satisfaction Inventory are used, a 
cornparison can be made between participants' current life satisfaction and 
their expected life satisfaction. 

(2) Specific factors of interest can be isolated for scoring according to the 
directions above. For example, program personnel might want to examine 
participants' expectafions for the factor "ability to handle problems." 
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HOW WILL YOU FEEL? 



Listed below are several factors that might influence your expectations about the 
quality of your life in the near future. 

In the IMPORTANCE column, Column 1, indicate how important each factor is to 
you by using the following scale: 

1 = Unimportant 

2 = Somewhat important 

3 = Very important 

In the EXPECTED SATISFACTION IN ONE YEAR column, Column 2, indicate 
how satisfied you expect to be with each factor one year from novr' by using the following 
scale: 

1 = Expect to he dissatisfied 

2 = E^Dect to he somewhat satisfied 

3 = Expect to he very satisfied 



Column 1 Column 2 



FACTOR 


IMPORTANCE 
TO QUALITY 
OF LIFE 


EXPECTED 
SATISFACTION 
IN ONE YEAR 


Your health 






Your physical appearance 






Your occupation (e.g., job, school, homemaking) 






Your ability to get along with people 






Your relationships with friends 






Your relationships with family 






Your sexual relationships 






Your spiritual life 






Your ability to handle problems 






Your financial condition 






Your leisure-time activities (e.g., hobbies, 
volunteer work, exercise program) 






Your success in achieving personal goals 







KEEPING YOUR COOL 

This affective measure assesses participants' perceived ability to manage stress. This 
measure is appropriate for adults. 

PURPOSE 

Information about participants* self-efficacy in being able to manage stress may be useful 
for the following reasons: 

9 Adnunistration of this measure at the beginning of the program may 
provide needs assessment information. For example, results of this 
measure may indicate that participants have a low perceived ability to 
manage stress and therefore may need training in this area. 

• When this measure is administered prior to and following a program, 
it is possible to evaluate growth in participants' self-efficacy m coping 
with stress. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end of the pro-am. 
However, handbook users should be alert to concerns regarding the potential reactivity of 
affective measures. A measure is considered reactive if the experience of completing the 
measure prior to the program causes participants to react aifferently to the program. 
Handbook users should, therefore, carefully review each affective measure that they wish to 
use to determine its potential for making participants unduly sensitive to aspects of the 
program. If a measure is determined to be reactive, then program peisonnel should not 
administer that measure to all participants as a pretest and posttest. Instead, the measure 
could be administered to half of the program participants prior to program participation to 
determine participants' pre-program status. The measure could then be admim'stered to the 
other half of the participants after program participation to assess participants' 
post-program status. 

SCORING AND ANALYSIS 

Point values are assigned to responses as follows: 
Definitely Yes = 5 

Probably Yes = 4 

Maybe = 3 

Probably No = 2 

Defim'tely No =1 

TTiis inventory can be scored by adding the point values of all responses across 
participants and dividing this total by the number ot responses. Blank items should not be 
counted in the number of responses. The maximum attamable score of 5 points indicates a 
strong perceived abiliW to manage stress across a variety of stressful situations. A minimum 
score of 1 indicates a lack of perceived ability to manage stress across a variety of stressful 
situations. 




KEEPING YOUR COOL 



This survey describes various situations when people might 
feel stress* Put a check to show how sure you are that you 
could cope witii each situation* By coping mean reducing 
the stress you experience in the situation. 



Couid you cope with the 
situation if * « , 



Definitely Probably 
Yes Yes 



Probably Definitely 
Maybe No No 



2. 



you are trying to 
concentrate, but you are 
constantly interrupted? 

you have to do a very boring 
task? 

3. you have been thinking 
about someone who hurt you 
in the past? 

4. you lose your job? 

5. you are stuck in heavy traffic? 

6. you lose something that is 
important to you? 

?• someone you are close to is 
angry with you? 

8- you have taken on more than 
you can do? 

9. someone in your family is 
very ill? 

10. you are late for an important 
appointment? 
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Keeping Your Cool, p. 2 



Defiiiitely Probably Probably Definitely 

Yes Yes Maybe No No 



Could you cope With the 
situation if.. . 



IL your closest friend has 
moved away and you feel 
lonely? 

12. you find out that you have a 
serious illness? 

13. you saw someone being 
robbed and keep thinking 
that it could happen to you? 

14. people keep asking you to 
do things you don't have 
time to do? 

15. a close friend of yours dies? 

16. you are moving to a new 
city? 

17. no matter how hard you 
have tried, you haven't been 
able to finish all your work? 

18. you have serious money 
problems? 

19. your relationship with a 
spouse or loved one ends? 

20. you just realized you have 
lost something that you need 
right away? 
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COULD YOU DEAL WITH IT? 



This affective measure assesses participants' ability to manage streets. This measure is 
appropriate for adolescents and preadolescents. 

PURPOSE 

Information about participants' self-efficacy in being able to manage stress ma> be useful 
for the following reasons: 

e Adniim'stration of this measure at the beginning of the program may 
provide needs assessment information. For example, results of this 
measure may indicate that participants have a low perceived ability to 
manage stress and therefore may need training in this area. 

• When this measure is administered prior to and following a program, 
it is possible to evaluate growth in participants' self-efficacy m coping 
with stress. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end of the program. 
However, handbook users should be alert to concerns regarding the potential reactivity of 
affective measures. A measure is considered reactive if the experience of completing the 
measure prior to the program causes participants to react differently to the program. 
Handbook users should, therefore, carefully review each affective measure that they wish to 
use to determine its potential for making participants unduly sensitive to aspects of the 
pro-am. If a measure is determined to be reactive, then program personnel should not 
administer that measure to all participants as a pretest and posttest. Instead, the measure 
could be administered to half of the program participa its prior to program participation to 
determine participants' pre-program status. The measure could then be administered to the 
other half of the participants after program participation to assess participants' 
post-program status. 

SCORING AND ANALYSIS 

■ Point values are assigned to responses as follows: 
Definitely Yes = 5 

Probably Yes = 4 

Maybe = 3 

Probably No =2 

Defim'tely No =1 

TTiis inventory can be scored by adding the point values of all responses across 
participants and dividing this total by the number oi responses. Blank items should not be 
counted in the number of responses. The maximum attainable score of 5 points indicates a 
strong perceived ability to manage stress across a variety of stressful situations. A minimum 
score of 1 indicates a lack of perceived ability to manage stress across a variety of stressflil 
situations. 
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COULD YOU DEAL WITH IT? 



The questions below ask about times when you might feel 
upset or nervous. Put a check to show how sure you are that 
you could deal with your feelings in each situation. 



Could you deal with it if... 



Definitely Probably 
Yes Yes 



Probably Definitely 
Maybe No No 



1. you had to help at home 
rather than be with your 

friends? ( ) 

2. you got a poor grade on a 

test? ( ) 

3. you kept getting more things 
to do and you were already 

busy? ( ) 

4. you didn't understand a 

lesson at school? ( ) 

5. you moved and didn't knov/ 
anyone at your new school? ( ) 

6. you didn't have enough time 
to answer all the questions 

on a test? ( ) 

7. your parents were angry 

with you? ( ) 

8. someone in your family was 

very sick? ( ) 

9. you thought that no one in 

your class liked you? ( ) 

10. you were home alone and 

had nothing to do? ( ) 
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Could You Deal With It?, p. 2 



Could you deal with it if.. 



Definitely Probably 
Yes Yes 



11. your family expected you to 
do too much? 

12. your closest friend moved 
away and you felt lonely? 

13. you couldn't join a club or 
team you wanted to? 

14. you lost something 
important? 

15. you had to sit quietly 
through something that was 
boring? 

16. you had too much 
homework to do in a short 
time? 

17. your parents were angry 
with each other? 

18. you didn't have enough 
money to spend? 



Probably Definitely 
Maybe No No 
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HOW WILL YOU COPE? 



This affective measure assesses participants* intention to use stress management 
techniques. Bothpositive and negative techniques for managing stress are included 
in the measure- This measure is appropriate for adults and adolescents. 

If this measure seems useful, you might also want to consider administering the 
Stress Management Checklist, which is a behavior measure assessing participants' 
actual use of stress management techniques. 

PURPOSE 

Information about participants' intention to use stress management techniques 
may be useful for the following reasons: 

• Administration of this measure at the beginning of the 
program may provide needs assessment information. For 
exami)le, results of this measure may show weak participant 
intention to use positive stress management techniques and 
thus indicate a need for instruction in the desirability of those 
techniques. 
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When this measure is administered prior to and follovrfng a 
program, it is possible to evaluate changes in participants' 
mtention to use positive stress management techniques. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end of the 
program. However, handbook users should be alert to concerns regarding the 
potential reactivity of affective measures. A measure is considered reactive if the 
experience of completing the measure prior to the program causes participants to 
react differently to the program. Handbook users should, therefore, carefully review 
each affective measure that they wish *^ use to determine its potential for making 

garticipants unduly sensitive to aspect the program. If a measure is determined to 
e reactive, then program personnel should not administer that measure to all 
participants as a pretest and jposttest. Instead, the measure could be administered to 
half of the program participants prior to program participation to determine 
participants' pre-program status. The measure could then be administered to the 
other half or the participants after program participation to assess participants' 
post-program status. 

SCORING AND ANALYSIS 

Items 2, 4, 10, 14, 16, 19, 22, 24, and 26 are commonly considered some of the 
negative ways for handling stress. For purposes of scormg, they will be labeled 
"negative items." The remaining items are positive coping techniques. They will be 
labeled "positive items." 

Total Score for Negative Items Marked YES or MAYBE 

1. Count the number of negative items that are marked either YES or MAYBE, 
(Ignore any blank or NO responses.) 
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2. Divide this total by the number of program participants. 

EXAMPLE: Imagine that there are 10 program participants. First, count all 
the times that these individuals marked either YES or MAYBE on the negative 
items (2, 4, 10, 14, 16, 19, 22, 24, and 26). Let's assume the total number of 
times was 55. Then, divide 55 by 10 participants to get an average score of 5.5. 

Scores can range from 0-9 with low scores indicating a group of participants who 
intend to use few negative coping techniques and high scores indicatmg the intention 
to use many negative techniques. 

Subtotal Score for Negative Items Marked YES or MAYBE 

1. Count only the number of negative items (2, 4, 10, 14, 16, 19, 22, 24, and 26) 
that are marked YES. 

2. Divide this total by the number of times the negative items were marked YES 
or MAYBE. Multiply this number by 100 to obtain a percentage. 

EXAMPLE: For the same 10 individuals used in the example above, count 
the number of times they^ marked YES on the negative items. Let's assume the 
total was 35. Then, divide by the total number of times the 10 individuals 
marked either YES or MAYBE on the negative items. This number was 
already determined to be 55 in the previous example. Thus, divide 35 by 55 to 
find oui., of the negative strategies participants intend to use, what percentage 
are definitely intended to be used In this case, 35/55 is about 64%. Thus, of 
the negative strategies indicated, 64% are definitely intended to be used, and 
36% are less likely to be used 

Besides seeing an overall reduction in the number of negative coping strategies 
intended to be used, program evaluators will hopefully see a decrease in the YES 
percentage and an increase in the MAYBE percentage when the measure is 
admim*stered at the beginning and end of the program. 

Scoring for Positive Items Marked YES or MAYBE 

The positive responses for handling stress can be scored in the same way that the 
negative responses were handled. However, in this case, because of the larger 
number of positive iteins in the instrument, the possible range of scores is from 0-17, 
with low scores indicating the intention to use few positive coping strategies and high 
numbers indicating the intention to use many positive strategies. The determination 
of percentages for the YES and MAYBE columns can also be done with the positive 
strategies. In the case of positive strategies, it would be desirable to see a switch in 
the direction of a higher percentage of YES responses being reported. 

Note: When dealing with the scoring of positive stress management techniques, 
program evaluators should not be overly concerned about group scores that do 
not extend into the upper end of the range. It seems unlikely that even the most 
skilled positive stress management participants would intend to use all the 
positive coping strategies. Rather, individual participants may find a limited 
number of positive strategies that work well for them. 
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HOW WILL YOU COPE? 



Listed below are things that people might do when they feel 
stress. Think about how you will deal with any problems or 
stressful situations that you might face in the near future. Then 
put a check to show how likely you are to do each of the 
following. 



1. tell yourself positive things? ( ) ( ) ( ) 

2. drink more coffee? ( ) ( ) ( ) 

3. try to figure out exactly why you are upset? ( ) ( ) ( ) 

4. eat more? ( ) ( ) ( ) 

5. find humor in the situation? ( ) ( ) ( ) 

6. use relaxation techniques? ( ) ( ) ( ) 

7. think about the positive side of the 

situation? ( ) ( ) ( ) 

8. exercise more? ( ) ( ) ( ) 

9. talk about the situation with the people 

who are involved? ( ) ( ) ( ) 

10. smoke more? ( ) ( ) ( ) 

11. think through how you will handle the 

situation? ( ) ( ) ( ) 

12. try to get more information about the 

situation? ( ) ( ) ( ) 

13. make a plan of action and follow it? ( ) ( ) ( ) 

14. drink more alcohol? ( ) ( ) ( ) 

15. accept the situation if nothing can be done 

to change it? ( ) ( ) ( ) 



When you feel stress in the 
near future^ will you . . . 



Yes 



Maybe 



No 
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How Will You Cope?, p. 2 



When you feel stress in the 
near future^ will you . . . 



16. take drugs or non-prescription 
medications? 

17. take things one step at a time? 

18. talk about your feelings with family or 
friends? 

19. take your frustration out on other people? 

20. start learning new skills to help you deal 
with the situation? 

21. get advice from someone who can help 
you? 

22. take the blame for problems that aren't 
your fault? 

23. do other things to give your mind a rest 
from the situation? 

24. keep your feelings to yourself? 

25. consider several alternatives for handling 
the situation? 

26. sleep more? 



Yes Maybe No 
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WHAT WILL YOU DO? 



This affective measure assesses participants' intention to use stress management 
techniques. Both positive and negative techniques for managing stress are included 
ill the measure. This measure is appropriate for adolescents and preadolescents. 

If this measure seems useful, you might also want to consider administering the 
How You Deal with Your Stress measure, which is a behavior measure assessing 
participants' actual use of stress management techniques. 

PURPOSE 

Information about participants' intention to use stress management techniques 
may be useful for the following reasons: 

• Administration of this measure at the begirming^ of the 
program may provide needs assessment information. For 
exami)Ie, results of this measure may show weak participant 
intention to use positive stress management techniques and 
thus indicate a need for instruction in the desirability of those 
techniques. 

• When this measure is administered prior to and following a 
program, it is possible to evaluate changes in participants' 
mtention to use positive stress management techniques. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end of the 
program. However, handbook users should be alert to concerns regarding the 
potential reactivity of affective measures. A measure is considered reactive if the 
experience of completing the measure prior to the program causes participants to 
react differently to the program. Handbook users should, therefore, carefully review 
each affective measure that they wish to use to determine its potential for making 
articipants unduly sensitive to aspects of the program. If a measure is determined to 
e reactive, then program personnel should not administer that measure to all 
articipants as a pretest and jposttest. Instead, the measure could be administered to 
alf ^ of the program participants prior to program participation to determine 
participants' pre-program status. The measure could then be administered to the 
other naif or the participants after program participation to assess participants' 
post "program status. 

SCORING AND ANALYSIS 

Items 6, 10, 13, 15, 18, and 19 are generally considered some of the negative ways 
for handling stress. For purposes of scoring, they will be labeled "negative items." 
The remaining items are positive coping techniques. They will be labeled "positive 
items." 

Total Score for Negative Items Marked YES or MAYBE 

1 Count the number of negative items that are marked as YES or MAYBE. 
(Ignore any blank or NO responses.) 



ERLC 



2. Divide this total by the number of program participants. 

EXAMPLE: Imagine that there are 10 program participants. First, count all 
the times that these individuals marked either YES or MAYBE on the negative 
items (6, 10, 13, 15, 18, and 19). Let's assume the total number of times was 
32. Then, divide 32 by 10 participants to get an average score of 3.2. 

Scores can range from 0-6 with low scores indicating a group of participants who 
intend to use few negative coping techniques and high scores indicatmg the intention 
to use many negative techniques. 

Subtotal Score for Negative Items Marked YES or MAYBE 

1. Count only the number of negative items (6, 10, 13, 15, 18, and 19) that are 
marked YES. 

2. Divide this total by the number of times the negative items were marked YES 
or MAYBE. Multiply by number times 100 to obtain a percentage. 

EXAMPLE: For the same 10 individuals used in the example above, count 
the number of times they^ marked YES on the negative items. Let's assume the 
total was 11. Then, divide by the total number of times the 10 individuals 
marked either YES or MAYBE on the negative items. This number was 
already determined to be 32 in the previous example. Thus, divide 11 by 32 to 
find out, of all the negative strategies particwants intend to use, what 
percentage are definitely intended to be used In this case, 11/32 is about 34%. 
Thus, of the negative strategies indicated, 34% are definitely intended to be 
used:, and 66% are less likely to be used. 

Besides seeing an overall reduction in the number of negative coping strategies 
intended to be used, program evaluators will hopefully see a decrease m the lES 
percentage and an increase in the MAYBE percentage when the measure is 
administered at the beginning and end of the program. 

ScoriKg for Positive Items Marked YES or MAYBE 

The positive responses for handling stress can be scored in the same way that the 
negative responses were^ handled. However, in this case, because of the larger 
number of positive items in the instrument, the possible range of scores is from 0-15, 
with low scores indicating the intention to use few positive coping strategies and high 
numbers indicating the intention to use many positive strategies. Hie deiermination 
of percentages for the YES and MAYBE columns can also be done with the positive 
strategies. In the case of positive strategies, it would be desirable to see a switch in 
the direction of a higher percentage of YES responses being reported. 

NOTE: When dealing with the scoring of positive stress m- iiagement techniques, 
program evaluators should not be overly concerned about group scores that do 
not extend into the upper end of the range. It seems unlikely that even the most 
skilled positive stress management participants would intend to use all the 
positive coping strategies. Rather, individual participants may find a limited 
number of positive strategies that work well for them. 
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WHAT WILL YOU DO? 



Listed bdow are things that people might do when they feel 
stress. Think about how you will deal with any problems or 
stressful situations that you might face in the near future. Then 
put a check to show how likely you are to do each of the 
following. 



When you feel stress in the 
nearfuiure^ will you . . . 



Yes 



Maybe 



No 



1. tell yourself good things about your life? 

2. try to figure out exactly why you are upset? 

3. use relaxation skills? 

4. try to see the good side of the situation? 

5. exercise more? 

6. smoke cigarettes? 

7. think about how you will handle the 
situation? 

8. try to get more information about things 
that worry you? 

9. make a plan of action and follow it? 

10. have a drink (such as beer or wine)? 

11. accept things if nothing can be done about 
them? 

12. do other things for awhile to get your mind 
off the situation? 

13. get high? 

14. talk about your feelings with your family? 

15. take your anger out on other people? 
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What Will You Dv?, p. 2 



When you feel stress in the 
nearfuturey will you . . . 



16. start learning new skills to help you deal 
with the situation? 

17. ask others for help if you need it? 

18. take the blame for problems that aren't 
your fault? 

19. keep your feelings to yourself? 

20. do other things you like when you can't do 
what you want? 

21. think about some different ways to handle 
the situation? 



Yes 


Maybe 


No 


I. ; 


^ ; 


() 


() 


() 


() 


() 


() 


() 


( ) 


( ) 


() 


() 


() 


() 


() 


() 


() 
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PEOPLE IN YOUR LIFE 



This affective measure assesses participants' perceived support and assistance from 
people around them. This measure is appropriate for adults. 

PURPOSE 

Information about participants' perceived support from other people may be useful for 
the following reasons: 

e Administration of this measure at the beginning of the program may 
provide needs assessment information. For example, results of this 
measure may indicate that participants need assistance in 
strengthening their support network. 

• Wiien this measure is administered prior to and following a program, 
it is possible to evaluate changes in participants' perceptions of their 
support group. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end of the program. 
However, handbook users should be alert to concerns regarding the potential reactivity of 
affective measures. A measure is considered reactive if the experience of completing the 
measure prior to the program causes particijjants to react differently to the program. 
Handbook users should, therefore, carefully review each affective measure that they wish to 
use to determine its potential for making participants unduly sensitive to aspects of the 
program. If a measure is determined to be reactive, then program personnel should not 
administer that measure to all participants as a pretest and posttest. Instead, the measure 
could be administered to half of the program participants prior to program participation to 
determine participants' pre-program status. The measure could then be administered to the 
othei halt of the participants after program participation to assess participants' 
post-program status. 

SCORING AND ANALYSIS 

Point values are assigned to responses as follows: 
Definitely Yes = 5 

Probably Yes = 4 

Maybe = 3 

Probably No = 2 

Definitely No =1 

This inventory can be scored by adding the point values of all responses across 
participants and dividing this total by the number of responses. Blank items should not be 
counted in the number of responses. The maximum attainable score of 5 points indicates a 
high level of perceived social support. A minimum score of 1 suggests very weak perceived 
social support. 
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PEOPLE IN YOUR LIFE 



This survey is about the support and assistance people may 
receive from others* Put a check to show how sure you are 
that you receive the kind of support described in each item. 




1. Are there people you trust 
to give you advice? 

2. Are there people you feel 
comfortable contacting if 
you are lonely? 

3. Are there people who would 
go out of their way to give 
you a ride if you needed 
one? 

4. Are there people who would 
look after your home if you 
went away for awhile? 

5. Are there people who are 
willing to criticize you for 
your own good? 

6. Are there people who are 
proud of your 
accomplishments? 

?• Are there people who would 
help you move to a new 
home? 

8. Are there people who would 
lend you money if you 
needed it? 

9. Are there people with whom 
you share your deepest 
thoughts and feelings? 



Definitely Probably 

Yes Yes Maybe 



Probably Definitely 
No No 
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People in Your LifCj p. 2 



Definitely 
Yes 

10. Are there people who would 
help you if you had too 

much to do? ( ) 

11. Are there people you could 
stay with if you needed to 

get away for awhile? ( ) 

12. Are there people who help 
you feel good about 

yourself? ( ) 

13. Are there people who care 
deeply about you? ( ) 

14. Are there people you call if 
you want to go out on the 

spur cf the moment? ( ) 

15. Are there people who would 
run errands for you if you 

were sick? ( ) 

16. Are there people who 
accept you just the way you 

are? ( ) 

17. Are there people who give 

you physical affection? ( ) 

18. Are there people who are 
willing to help you make 
important decisions? ( ) 

19. Are there people who help 
you feel better when you are 
upset? ( ) 

20. Are there people who help 
you work out personal 
problems? ( ) 

21. Are there people who are 
supportive of you no matter 
what? ( ) 



Probably Probably Definitely 

Yes Maybe No No 
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THE PEOPLE YOU KNOW 



This affective measure assesses participants' perceived support and assistance from 
people around them. This measure is appropriate for adolescents and preadolescents. 



Information about participants' perceived support from other people may be useful for 
the following reasons: 

« Administration of this measure at the beginning of the program may 
provide needs assessment information. For example, results of this 
measure may indicate that participants need assistance in 
strengthening their support network. 

© When this measure is administered prior to and following a program, 
it is possible to evaluate changes in participants' perception of their 
support group. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end of the program. 
However, handbook users should be alert to concerns regarding the potential reactivity of 
affective measures. A measure is considered reactive if the experience of completing the 
measure prior to the program causes participants to react differently to the program. 
Handbook users should, therefore, carefully review each affective measure that they wish to 
use to determine its potential for making participants unduly sensitive to aspects of the 
progjram. If a measure is determined to be reactive, then program personnel should not 
administer that measure to all participants as a pretest and posttest. Instead, the measure 
could be administered to half of the program participants prior to pio^^cvm participation lo 
determine participants' pre-program status. The measure could then be administered to the 
other hall of the participants after program participation to assess participants' 
post-program status. 

SCORING AND ANALYSIS 

Point values are assigned to responses as follows: 
Definitely Yes = 5 

Probably Yes = 4 



This inventory can be scored by adding the point values of all responses across 
participants and dividing this total by the numbei ut responses. Blank items should not be 
counted in the number of responses. The maximum attainable score of 5 points indicates a 
high level of perceived social support. A minimum score of 1 suggests veiy weak perceived 
social support. 



PURPOSE 



Maybe 
Probably No 
Definitely No 



3 
2 



1 



112 



109 



THE PEOPLE YOU KNOW 



Listed below are some different kinds of support and help 
that people receive from others. Put a check to show how 
sure you are that you receive the kind of support described 
in each question. 



Definitely Probably 
Yes Yes 



Maybe 



Probably Definitely 
No No 



1. Are there people who would 
lend you anything if you 
needed it? 

2. Are there people who would 
help you if you had 
something that was too hard 
to do? 

3. Are there people who are 
proud of you? 

4. Are there people who love 
you? 

5. Are there people you can 
talk to about your problems? 

6. Are there people who like 
you just the way you are? 

7. Are there people who would 
help you make important 
decisions? 

8. Are there people who help 
you feel better when you are 
upset? 

9. Are there people you can 
talk to if you feel lonely? 

10. Are there people you can 
talk to about your feelings? 



() 



() 
() 
() 
() 
() 

() 

() 
() 
() 



() 



() 



() 



() 
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People You Know, p. 2 



11. Are there people who joke 
and Idd around with you? 

12. Are there people \/ho would 
take care of something of 
yours if you went away for 
awhile? 

13. Are there people who help 
you feel good about 
yourself? 



Definitely Probably 

Yes Yes Maybe 

() () () 

() () () 

() () () 



Probably Definitely 
No No 

() () 

() () 
() () 
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LIFE SATISFACTION INVENTORY 



This affective measure asks participants to indicate the importance of 12 factors 
that influence their equality of hfe. Participants weigh the importance of each factor 
and then indicate their level of satisfaction. This measure is appropriate for adults. 

If this measure seems useful, you might also want to consider the How Will You 
Feel? measure, which is another affective measure assessing participants' perceptions 
about quality of life. The How Will You Feel? measure asks for expected satisfaction 
levels rather than current satisfaction levels. 

PURPOSE 

Information about participants' life satisfaction may be useful for the following 
reasons: 

• Administration of this measure at the beginning^ of the 
program may provide needs assessment information. For 
example, results of this measure may show that participants 
have low perceptions of their quality of life, thus emphasizing 
their need for mstruction in stress management techniques. Ii 
participants feel that thejr are better able to handle stress, their 
life satisfaction may also increase. 

e When this measure is administered prior to and following a 
program, it is possible to evaluate changes in participants' 
perception of their quality of life in twelve distinctive areas. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end of the 
program. However, handbook users should be alert to concerns regarding the 
potential reactivity of affective measures.^ A measure is considered reactive if the 
experience of completing the measure prior to the program causes participants to 
react differently to the program. Handbook users should, therefore, carefully review 
each affective measure that they wish to use to determine its potential for making 
participants unduly sensitive to aspects of the piogram. If a measure is determined to 
be reactive, then program personnel should not administer that measure to all 
participants as a pretest and posttest. Instead, the measure could be administered to 
half of the program participants prior to program participation to determine 
participants' pre-program status. The measure could then be administered to the 
other naif ot the participants after program participation to assess participants' 
post-program status. 

SCORING AND ANALYSIS 

This inventory can be scored in two ways. One procedure relies on the 
SATISFACTION responses only, providing an easily obtained group estimate of 
participants' current life satisfaction. 

A second procedure combines the IMPORTANCE responses with the 
SATISFACTION responses. This method provides a weighted assessment of life 
satisfaction based on the IMPORTANCE rating. 
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Procedure 1— Satisfaction Responses Only 

1. Add the point values of all the responses in Column 2 for all participants. 
(Ignore blanks; that is, do not count them when you count the number of 
responses.) 

2. Divide this total by the total number of responses. 

EXAMPLE: Imagine that there are 10 program participants. First, add up all 
the numbers in Column 2 for all participants. Let's say the total is 190. Divide 
this total by the total number of responses from all participants. We will use 
110 responses for this example. (The total possible number of responses is 120 
if there are no blanks in Column 2.) Then divide 190 by 110 responses to get 
an average score of about 1.7. 

The maximum total SATISFACTION score is 3.0; the minimum score is 1.0. 
Scores near 3.0 indicate that participants as a group have a high level of life 
satisfaction. Scores near 1.0 indicate that participants are dissatisfied. 

Procedure 2— Satisfaction Weighted by Importance Responses 



1. 



Count the number of items rated "very important" (3) in Column 1. 



2. Total the SATISFACTION ratings in Column 2 for the items counted in step 

3. Divide the total in step 2 by the total in step 1. 

EXAMPLE: For the same 10 participants used in the example above, count 
the number of items that they rated as '"very important'' in Column 1. Of the 
120 responses possible, let's say there were 43 responses marked with a 3. For 
those same 43 responses, move over to Column 2 and add up the ratings given 
to them. Let's assume a total of 116. Then, divide 116 by 43 for a 
SA TISFACTION average score of 2.7 which is weighted by participants' '"very 
important" rating. 

Again, the maximum attainable score is 3.0; the minimum is 1.0. Scores close to 
3.0 indicate that participants are very satisfied with those factors rated "very 
important." 

This same procedure can also be followed to determine participants' satisfaction 
for those items rated "somewhat important." 

NOTES: 

(1) If both this measure and the How Will You Feel? measure are used, a 
comparison can be made between participants* current life satisfaction and 
their expected satisfaction. 

(2) Specific factors of interest can be isolated for scoring according to the 
directions above. For example, you might want to examine participants* 
satisfaction with the factor, "ability to handle problems." 



ERIC 



^ lie 

113 



LIFE SATISFACTION INVENTORY 



Listed below are several factors that might influence your overall life satisfaction. 

In the IMPORTANCE column, Column 1, indicate how important each factor is to 
you by using the following scale: 

1 = Unimportant 

2 = Somewhat important 

3 = Very important 

In the SATISFACTION column, Column 2, indicate how satisfied you are with each 
factor by using the following scale: 

1 = Dissatisfied 

2 = Somewhat satisfied 

3 = Very satisfied 



Column 1 Column 2 



FACTOR 


IMPORTANCE 


SATISFACTION 


Your health 






Your physical appearance 






Your occupation (e.g., job, school, homemaking) 






Your ability to deal with people 






Your relationships with friends 






Yr^"ir relationships with family 






Your sexual relationships 






Your spiritual life 






Your ability to handle problems 






Your financial condition 






Your leisure-time activities (e.g., hobbies, 
volunteer work, exercise program) 






Your success in achieving personal goals 







117 



ARE 70U HAPPY? 



This affective n.easure assesses participants' happiness with several factors that may 
influence their o^ e:all quality of life. Uns measu*a is appropriate for adolescents and 
preadolescents. 

PURPOSE 

Information about participants' life satisfaction may be useful for the following reasons: 

o Adniinistration of this measure at the beginning of the program may 
provide needs assessment information. For example, results of this 
measure may show that participants have low perception of their 
quality of life, thus emphasizing their need for instruction in stress 
management techniques. If participants feel that they are better able 
to handle stress, their life satisfaction may also increase. 

• When this measure is administered prior to and following a program, 
it is possible to evaluate changes in participants' perception of their 
quality of life in eight areas. 

PROCEDURES 

This instrument can be administered both at the beginning and a* the end of the program. 
However, handbook users should be alert to concerns regarding the potential reactivity of 
affective measures. A measure is considered reactive if the experience of completing the 
measure prior to the program causes participants to react differently to the program. 
Handbook users should, therefore, carefully review each affective measure that they wish to 
use to determine its potential for making participants unduly sensitive to aspect^ of the 
program. If a measure is determined to be reactive, then program personnel should ::uc 
administer that measure to all participants as a pretest and posttest. Instead, the measure 
could be administered to half of the program participants prior to program participation to 
determine participants' pre-program status. The measure could then be administered to the 
other half of the participants after program participation to assess participants' 
post-program status. 

SCORING AND ANALYSIS 

Point values are assigned to responses as follows: 
Definitely Yes = 5 
Probably Yes = 4 
Maybe = 3 

Probably No =2 
Definitely No =1 

This inventory can be scored by adding the point values of all responses across 
participants and dividing this total by the number of responses. Blank items should be 
Ignored and not counted when determining the number of responses. The maximum score 
attainable is 5; the minimum is 1. Scores near 5 indicate a high degree of life satisfaction 
while scores near 1 indicate a low degree of life satisfaction. 
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ARE YOU HAPPY? 



Listed below are several things that might be important in 
your life. Put a check under the answer that tells how happy 
you feel about each part of your life. 



Are you happy with ♦ . . 



L your health? 

2. the way you look? 

3. your work at school? 

4. the way you get along with 
people? 

5. your friends? 

6. your family life? 

7. the amount of money you 
have? 

8. what you do in your free 
time (hobbies, sports, etc,)? 



Definitely Probably 
Yes Yes 



() 
() 
() 

() 
() 
() 

() 

() 



Probably Definitely 
Maybe No No 
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IDEAS ABOUT DECISIONS 

This affective measure assesses participants' belief in the value of careful 
decision-making. This measure is appropriate for adolescents and 
preadolescents. 

PURPOSE 

Information about participants' belief in careful decision-making may be 
useful for the following reasons: 

• Administration of this measure at the beginning of the 
program may provide needs assessment information. 
For example, results of this measure may indicate a 
need for strengthening participant's appreciation for 
careful decision-making in deal^'ng with stress-related 
situations in their lives. 

• When^ this measure is administered prior to and 
following a program, it is jjossible to evaluate growth 
in {participants' appreciation for care^al decision- 
malcing. 

PROCEDURES 

This instrument can be administered both at the beginning and at the end 
of the program. Ho\vever, handbook users should be alert to concerns 
regard; ^ .he p)otential reactivity of affective measures. A measure is 
considered reactive if the experience of completing the measure prior to the 
program causes participants to react differently to the pyogram. Handbook 
users should, therefore, carefully review each affective measure that they wish 
to use to determine its potential for making participants unduly sensitive to 
aspects of the program. If a measure is determined to be reactive, then 
program personnel should not administer that measure to all participants as h 
pretest and posttest. Instead, the measure could be administered to half of the 
program participants prior to program participation to determine 
participants' pre-program status. The measure could tnen be administered to 
the otner half of the participants after program participation to assess 
participants' post-program status. 
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SCORING AND ANALYSIS 



Point values are assigned to responses as follows: 



Item 


Strongly 




Not 




Strongly 


No. 


Agree 


Agree 


Sure 


Disagree 


Disagree 


1 


5 


4 


3 


2 


1 


2 


1 


2 


3 


4 




3 


5 


4 

• 


3 


2 


1 
1 


4 


1 


2 


3 


4 


5 


5 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


7 


1 


2 


3 


4 


5 


8 


5 


4 


3 


2 


1 


9 


5 


4 


3 


2 


1 


10 


5 


4 


3 


2 


1 



This inventory can be scored by adding the point values of all responses 
across participants and dividing this total by the number of responses. Blank 
items should not be counted in the number of responses. The maximum 
attainable score of 5 points indicates a strong belief in the importance of 
making decisions carefully. 
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IDEAS ABOUT DECISIONS 



The sentences below are about making decisions. For each 
sentence, place a check to show how much you agree or 
disagree with the sentence. 



Strongly 
Agree 



Agree 



Not Strongljr 
Sure Disagree Disagree 



1- It is worth the time it takes 
to make decisions carefully. 

2. People should go with their 
first ideas when making 
decisions. 

3. People are happier with 
their decisions when they 
take the time to make them 
carefully. 

4. Spending a lot of time to 
make careful decisions is too 
difficult. 

5. Making careful decisions 
takes too much time. 

6. When making decisions, 
people should do what they 
feel, not what they think. 

7. People maLe equally good 
decisions no matter how they 
arrive at them. 

8. People who make quick 
dc nsions are usually 
disappointed with them later. 

9. Peoj-^e should take time to 
make decisions carefully. 

10. It is easy to make decisions 
carefully. 
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CHAPTER FOUR 
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Locally Conducted Psychometric Studies 



As described in Chapter One, the first step in using the newly developed handbook 
measures to examine program effectiveness is to select those measures that match program 
goals. However, evaluators cannot assume that a measure that appears to assess a desired 
program outcome will produce valid data about that outcome. When evaluators use a 
measure, they first want to determine the technical quality of that measure to ensure that 
any conclusions drawn about a program's. effects are warranted. The purpose of this chapter 
is to assist evaluators in conducting validation studies for those handbook measures chosen 
for use in program evaluation. 

Determining the Teclinical Quality of Measuring Devices 

The degree to which a measuring instrument yields scores from which one can make 
legitimate inferences is referred to as validity. Tests are not valid or invalid. Rather, it is the 
inferences made, based on test results, that are valid or invalid. It is, therefore, technically 
accurate to focus on the validity of score-based inferences rather than the validity of a 
particular measuring device. 

The concept of validity is highly dependent on the particular way in which a measuring 
instrument will be used. For example, a measure of the use of coping skills to deal with 
stre s may permit a valid inference regarding the number of different skills that program 
participants use, but may yield invalid inferences regarding the frequency with which 
participants use each skill. Fuithermore, a test may yield valid inferences for a particular 
purpose with one population bui invalid inferences for the same purpose with a different 
population. Thus, because validity varies on the basis of purpose and population, it is most 
appropriate to examine validity in the setting in which a measure will be used. 

A second factor in determining the technical quality of a measurement instrument deals 
with the extent to which the instrument produces reliable, that is, consistent results. 
Because the newly developed handbook measures have been subjected only to small-scale 
field tests, no reliability data are currently available. It is hoped that handbook users will 
conduct their own reliability studies and share those results with the Centers for Disease 
Control. In this way, results can be compiled over time and, subsequently, provided to 
handbook users. Procedures for evaluating the reliability of the handbook measures will be 
presented following a discussion of local validation approaches. 

Categories of Validity Evidence 

There are three major types of evidence regarding validity. These include content-related 
evidence of validity, criterion-related evidence of validity, and construct-related evidence of 
validity. The procedures for securing each type of validity evidence will be described below. 

Content-related evidence of validity. Content-related evidence of validity involves the 
carefiil reviev^ of a measure's content by Individuals identified as experts in the content area 
being assessed. This type of validity evidence is particularly important for measures 
designed to assess examinees' knowledge and skills. To secure positive content-related 
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validity, the measure must include only those items that correspond to the content area 
being assessed and its items must address all important facets of that content area. The 
systematic, expertise-rooted procedures used to develop the handbook's instruments helped 
to ensure that appropriate content was built into the measures. Subsequent reviews by 
external experts confirmed that the measures are, indeed, focused on suitable content. 
These development procedures and the role of expert advisors in the project are described 
in the handbook's preface. 

If there are questions regarding the suitability of the content in any of the handbook's 
measures, content-related validity can be examined by assembling a panel of experts who 
can judge the suitability of a measure's content for the specific program-evaluation purpose 
for which the measure is to be used. A panel of approximately ten knowledgeable 
individuals can be asked to review the measuring instrument's items, one-by-one, and 
render independent yes/no judgments regarding the appropriateness of each item*s content 
(in relationship to the inference that the program evaluators wish to make on the basis of 
the measure). In addition, panelists can be asked to determine whether any important 
content has been omitted from the measure. For example, if a knowledge measure such as 
Facts About Stress is being reviewed, panelists might be asked to first think of all the 
important facts about stress that program participants must know and then to indicate the 
percentage of those facts that are present in the measure being reviewed. This 
straightforward indication of a measure's content representativeness, when coupled v/ith 
judgments regarding the content appropriateness of a measure's items, can yield important 
content-related evidence of validity for a measure.* 

Criterion-related evidence of validity. Criterion-related evidence of validity requires that a 
measure be checked against an independent criterion. The independent criterion or 
standard should be one that the measure would be expected to predict. Criterion-related 
validity is most important for the handbook measures In the areas of behavior and intention. 
In the area of behavioral self-reports, for example, criterion-related validity would focus on 
the degree to which the self-reports reflect actual behavior. So, for example, 
criterion-related validity for a self-report instrument designed to measure the use of coping 
skills would be secured by correlating responses on this instrument with observations (by 
others) of the extent to which the skills were actually being used. 

External criterion measures, such as observations, while often more accurate measures of 
behavior than self-reports, are extremely costly and time consuming to use. Thus, although it 
may be possible to use such criterion measures in a one-time validity study, they typically 
will not eliminate the need for self-report instruments in routine program evaluations. The 
general procedure for conducting a criterion-related validity study is shown in Figure 4.1. 

A correlation of approximately 0.50 or higher between the measure and criterion would 
indicate that the new measure is predictive of the external criterion measure and, therefore. 



For additional Information about how to conduct content-related validation studies, see Annotated 
Bibliography Nos. 18, 23, 27, and 35 
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Select a criterion 
against which to 
compare the measure 
to be validated. 



Obtain scores on the 

measure and the 
criterion for a group of 
participants* 



Correlate the scores 
from the measure and 
the criterion* 



Figure 4.1: Procedure for conducting criterion-related validity studies 

is measuring what it is intended to measure. A low correlation would call into question the 
self-report mstrument as a measure of the behavior of interest. 

Each criterion-related validity study must be specifically designed for the particular 
measure being examined and the purpose for which it will be used. For example, imagine 
that an evaluator wanted to examine the criterion-related evidence of validity for the 
handbook's measure entitled How W:I1 You Cope? The evaluator must first identify an 
appropriate criterion measure. How is a program evaluator likely to use an intention 
measure? The most likely use would be to employ it as a pvo^iy measure foreshadowing a 
program's effect on the future behavior of participants. That is, will program participants 
continue to use coping skills in the future? Thus, an appropriate criterion measure mi^t be 
the reported use of coping skills several months following the program. 

To assemble criterion-related e\idence of validity for the intention measure, a program 
evaluator could administer the intention measure at the end of the program to a group of at 
least 30 participants (or repeat this process each session until responses from at least 30 
participants are obtained) and obtain completed self-report surveys several months later 
regarding participants' use of coping skills. Once both measures are collected for every 
individual, a correlation could be computed between the strength of intention for using a 
given coping behavior and whether the coping skill was being used following the program. 
Thus, the criterion-related validity study would examine whether the intention mi asure was, 
in fact, predictive of later behavior. A measure that can serve as a meaningtui pro;^' fo. 
participants' future behavior can prove highly useful in the evaluation of a program's impact 
on participants.* 

Construct-related evidence of validity. The final type of validity evidence to be reviewed, 
construct-related evidence of validity, is particularly important for those handbook 
measures that do not have a clear criterion against which they can be evaluated. Such 
measures include the attitudinal and affective measures such as Keeping Your Cool, a 
measure that examines an individual's perceived ability to cope with stressful situations, 
Construct-related validity involves the gradual accumulation of data regarding what a test 
measures. Three strategies aie customarily used to secure construct-related evidence of 



For additional information about the design and analysis of criterion-related validity studies, see 
Annotated Bibliography Nos. 18, 23, 27, and 35. 
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The Measure Being 
Reviewed 



Similar Measures 



Strong, Positive 
Relationships 



Dissimilar Measures 



Weak or Negative 
Relationships 



Fi^re 42: Correlations between measures assessing similar/dissimilar attitudinal dimensions 

validity for a measure. First, in the related-measures strategy, predictions can be tested about 
the extent to which the measure of interest is conelated with other measures. For example, 
perceived ability to ccpe should be positively related to other measures aimed at assessing a 
similar attribute but should show reduced correlations with measures tapping different 
attitudinal dimensions. Thus, other existing measures can be conelated v/ith the measure of 
interest to help clarify what is being measured. 

If the correlations are consistent with the prior predictions, then construct-related 
evidence of validity has been obtained to support the defensibility of inferences based on 
the measure's use. Figure 4.2 illustrates the anticipated correlations between the measure of 
interest and other similar and dissimilar measures. 

A second approach to examining construct-related validity involves predictions about 
group differences and is referred to as a differential-populations strategy. For this procedure, 
two or more groups are identified which are expected, based on other characteristics, to 
perform differently on the measure of interest. For example, the two groups might consist of 
individuals who work in high-pressure jobs versus those who do not. If the anticipated 
performance difference between the two groups is not obtained, it wolM raise the question 
as to whether the test was measuring what it was thought to measure. 

A third strategy for securing construct-related evidence of validity is referred to as an 
intervention strategy because it involves the use of interventions such as training programs. 
For instance, a measure examined via this strategy could be administered to a group of 
participants before and after a "proven ' stress management training program. If a difference 
in participants' scores on the measure is not observed, then the consiruct-related evidence 
of validity regarding the measure being reviewed is not supportive of the measure's use. 
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Construct-related evidence of validity is never based on a single study. Instead, 
consideration of a variety of studies, employing multiple validation strategies such as those 
described here, mil help provide greater clarification regarding the appropriateness of using 
a ghf-en measuring instrument.* 

Types of Reliability 

A second characteristic of a defensible measurement instrument is the reliability or 
consistency with vi^hich it measures. The reliability of a test can be examined in three distinct 
ways. These include test-retest reliability, alternate-forms reliability, and internal 
consistency. Each of these approaches will be described below. 

Test-retest reliability. Test-retest reliability (also referred to as stability reliability) 
examines the extent to which a measurement instrument is consistent over testing occasions. 
That is, will an individual who received a particular score on one testing occasion receive a 
similar score on a different testing occasion. Typically, to secure test-retest reliability 
information, an instrument is administered once to a group of individuals (30 or more). The 
same instrument is then administered again under similar cojiditions to the same group of 
individuals approximately two to four weeks later. Individuals' scores from the two 
admim'strations are then correlated. The higher the correlation, the greater the stability of 
measurement over time. Short tests, or other tests that are likely to be easily remembered, 
may result in an overestimate of reliability if participants recall their answers, and hence, 
respond similarly on the second testing occasion. 

Alternate-forms reliability. The knowledge and skill measures in this handbook have two 
forms that may be used for a pretest to posttest comparison. The administration of one form 
for the pretest and the other form for the posttest is desirable because the pretest may 
sensitize participants to pay more attention to those issues included on the pretest than to 
other equally important issues. However, to draw defensible conclusions based on the use of 
two different forms at pretest and posttest, the forms must be equivalent. 

To examine alternate-forms reliability, it is necessary to administer both forms to the 
same group of individuals. The scores from the two forms can then be correlated. High 
correlations indicate that the same conclusions would be drawn about an individual or group 
of participants regardless of which of the two forms had been used. Thus, there would be 
reliable or consistent measurement across alternate forms. A high alternate-forms reliability 
coefficient does not guarantee that the forms are perfectly equidifficult. If the two forms are 
not of equal difficulty, that is, participants perform consistently better on one form than the 
other, it would still be possible to obtain high between-forms correlations. Thus, it is 
important to be attentive to mean scores on the two test forms. It is also permissible to use 
p-values (the percent of examinees getting each item correct) to xeassign items to forms so 



* For additional information about how to conduct construct-related validity studies, see Annotated 
Bibliography Nos. 18, 23, 27, and 35. 
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that they are more equidifficult. After the redistribution of items, a second alternate-forms 
reliability study should be conducted. 

Handbook users should not assume equivalerxce or equidifficulty for the multiple forms 
provided in this handbook. Until alternate-forms reliability and test difficulty are examined, 
the measures should be used in a design such that half of the participants take Form A as a 
pretest and Form B as a posttest while the other half take Form B as a pretest and Form A 
as a posttest. This counterbalancing technique eliminates the possible influence of one form 
being more difficult than the other. 

Internal consistency. Internal consistency examines the extent to which the instrument 
measures a single or related set of constructs. The higher the internal consistency, the 
greater the homogeneity of items on the test A test thought to measure a single attitudinal 
dimension should have relatively high internal consistency reliability. Procedures for 
calculating internal consistency include split-half reliability, Kuder-Richaidson formulas, 
and Cronbach's Alpha. The split-half reliability coefficient L calculated by administering the 
tc >t to a group of at least 30 participants and then correlating scores from the odd versus the 
even items. A correction for test length must then be made using the Spearman-Brown 
formula. The split-half procedure is very similar to alternate-forms reliability in that two 
"forms'* are correlated by separating the odd and even items. Kuder-Richardson formulas 
for internal consistency provide an estimate of the average of all possible split-halves. These 
formulas, like Spearman-Brown, require that test items be binary-scored, that is able to be 
scored as right or wrong. Cronbach's Alpha is identical to Kuder-Richardson for binary 
scored items but can also be used for items that yield responses to which several points can 
be assigned, such as items on Keeping Your Cool. 

Not all forms of reliability need to be computed for every test. For example, 
alternate-forms reliability would be computed only for those measures that have two forms. 
Internal consistency estimates would not be appropriate for multidimensional measures. 
Test-retest reliability is appropriate for most measures but often presents pragmatic 
problems due to the need to retest the same individuals.* 

Groups and Individuals 

The valid:*^ and reliability procedures reviewed here were originally developed to 
examine the quality of tests used for individual assessment purposes. In contrast, the 
recommended use of the handbook measures is to perform group analyses for program 
evaluation. Thus, the appropriate reliability issue is whether scores for a group of individuals 
are relatively consistent. Similarly, the validity issue is whether changes in scores for a group 
of individuals are reflective of changes in the group's knowledge, skills, affect, or behavior. 
Because group scores are more stable than individual scores, the procedures outlined above 
are likely to underestimate the reliability and validity of the measures when used for 



For additional Information about how to examine the reliability of measurement instruments, sec 
Annotated Bibliography No, 18, 19, 23, 27, and 35. 
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program evaluation. Practically speaking, a measurement instrument with a lower reliability 
or validity coefficient would be acceptable when used for group rather than individual 
diagnosis. For example, Salvia and Ysseldyke (1981, p. 98) have recommended the following 
minimum standards for alternate-forms reliability: 

.60 - when scores are reported for groups 

.80 - when scores are used for individual screening 

.90 - when scores are used for importatit educational decisions for individuals 
Thus, standards for acceptable i eliability and validity vary depending on the purpose for 
using a particular measure. However, minimal levels for each are critical for making sound 
decisions about a program. With a little creativity and effort, studies of reliability and 
validity can often be integrated into the ongoing operation of a program. 

In addition to providing a brief overview, the major purpose of this' chapter was to 
encourage handbook users to conduct local reliability and validity studies and to consider 
the involvement of a measurement specialist or the use of appropriate references in 
designing such studies. As suggested at the outset of the chapter, if such local studies are 
carried out, results should be forwarded to the Centers for Disease Control (Attention: 
Dr. Diane Orenstein, Project Officer, Center for Health Promotion and Education, Centers 
for Disease Control, 1600 Clifton Road N.E., Atlanta, GA 30333). This information will be 
shared with future handbook users. 
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Appendix A 

AMPLIFIED CONTENT DESCRIPTORS* 



FACTS ABOUT STRESS 
(Aduit/Adolescent Measure) 

LEARNING ABOUT STRESS 
(Adolescent/Preadolescent Measure) 

The Nature of Stress 

1. Stress is the physiological and psychological response to any demand made on the 
individual. TTie sources of these demands are called stressors. 

2. Some degree of stress is essential for life. 

3. Stress is harmful when it is too extreme and/or lasts too long. 

4. The best level of stress is the amount that improves a person's performance without 
producing harmful side effects. 

5. Stress is a function of the environment and how one views and acts upon the 
environment. 

6. Frequently, the stress produced by a situation depends more on the person's 
perception of the situation than on the situation itself 

7. People react to physical stressors, psychosocial stressors, and symbolic stressors. 

The Nature of Stressors 

8. Heat, cold, injury, and physical exertion are physical stressors. 

9. Unprotected exposu-^, to loud noise and bright light is stressful. 

10. Frustration, crowding, overload, deprivation, noise, and life changes are psychosocial 
stressors* 

11. Frustration occurs because individuals lack the ability to take necessary actions or 
their actions are blocked by external obstacles. 

12. The amount of stress individuals feel when in a crowd depends on how much control 
they think they have in the situation and on their cultural background. 



The amplified content descriptors are not exhaustive accounts of stress management content. 
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13. Overload occurs when the demands on a person exceed that individual's ability to 
meet those demands. 

14. Deprivation occurs when individuals receive too little meaningful stimulation. 

15. Too little stimulation may be as stressful as too much. 

16. Noise can be stressful when it is unpredictable and inappropriate. 

17. Marriage, retirement, vacation, loss of a job, and the death of someone close are 
examples of life changes. 

18. Both favorable and unfavorable life changes require adjustment and, therefore, can 
be stressful. 

19. Symbolic stressors are the symbolic or imagined threat of physical or psychological 
stressors. 

20. Unwarranted fears and unpleasant thoughts or memories ar-; symbolic stressors. 

21. Thinking about an unpleasant event can cause an indi^/idual as much stress as actually 
experiencing the event. 

Personality Factors 

22. An individual's reaction to stressors is determined by that person's prior experiences, 
attitudes, values, and perception of the stressor. 

23. Thinking of oneself as helpless and worthless can lead to increased stress. 

24. An individual who sees problems as worse than they are is likely to be stressed most 
of the time. 

25. The Type A personality, which is associated with stress, is marked by excessive 
competitive drive, aggressiveness, and a sense of time urgency. 

26. One of the most common traits of the Type A personality is trying to think or do two 
or more things at the same time. 

Environmental Factors 

27. The most stressful situations are usually those over which individuals feel they have 
little control. 

28. An individual's expectation about a stressful event can influence the individual's 
stress level. 

29. An individual's stress level can increase if that individual receives no information ui 
false information about a potentially stressful event prior to its occurrence. 

Physiology of Stress 

30. The physiological responses to stressors occur automatically, without much conscious 
thought. 

31. Stress prepares the individual for physical activity. 



32. The physiological responses to stressors may occur in three stages. 

33. The first stage of responses include increased heart rate, increased blood pressure, 
increased rate of respiration, increased muscle contraction, increased activity of most 
hormones, and decreased digestive activity. 

34. If stress continues, the body enters the second stage in which the body adapts to the 
stressor and many body systems return to a normal level of functioning. 

35. If stress continues for a long time, the body enters the third stage in which exhaustion 
and disease may occur. 

The Effects of Stress 

36. Stress may lead directly and indirectly to illness. 

37. Stress may lead indirectly to illness by decreasing the body's ability to defend itself 
against disease. 

38. Hormones released under stress remain in the body for several hours and may lessen 
the body's ability to fight infection. 

39. Arthritis and cancer may be indirectly related to stress. 

40. Stress can lead directly to illness by causing the fatigue or failure of organ systems in 
the body. 

41. The type of illness from stress that an individual develops may depend on that 
person's heredity and prior experiences, and on which organ system is the weakest. 

42. Stress is a direct cause in the development of some muscular problems, ulcers, and 
high blood pressure. 

43. Headaches, backaches, posture problems, and spasms of the digestive tract are 
muscular problems that can be caused by prolonged muscle contraction due to stress. 

44. Many headaches are caused by prolonged contraction of the muscles of the head and 
neck. 

45. Muscles that are constantly contracted can cause a person to feel anxious. 

46. Muscles that are constantly contracted lead to increased stress. 

47. Constant arousal due to stress can cause an individual's blood pressure to remain at a 
high level. 

48. Although it usually lacks obvious symptoms, high blood pressure damages the heart, 
liver, kidneys, and other organs. 

49. Whether an individual's personality and behavior directly cause heart disease is a 
matter of dispute. 

50. The Type A personality is associated with heart disease. 

51. A person under stress may feel anxious, frustrated and/or worried. 

52. A person under stress may feel confused or unable to concentrate. 

53. Severe stress may make people accident-prone. 
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54. A person under stress may be insensitive toward other people. 

55. A person under stress may not be able to perform tasks as well as usual. 

56. A person under stress may return to old habits even though tncy may not be 
appropriate to the present situation. 

Stress-Related Aspects of Diet 

57. Both caffeine and sugar may be stressful for some people. 

58. Caffeine is present in coffee, tea, chocolate, some soft drinks and some 
over-the-counter medications. 

59. Excessive stress may increase the rate at which one's body uses up certain B vitamins 
and Vitamin C. 



136 135 



COPING WITH STRESS 
(AdultM aolescent Measure) 

WAY? ro LOWER STRESS 
(Ado'^^scent/Preadolescent Measure) 

General Information 

1. Individuals cannot avoid all stress, but they can learn to cope with stress. 

2. Individuals should become aware of how they experience stress in their bodies. 

3. Individuals should identify the environmental situations that prompt their stress, 

4. Even if individuals cannot change the nature of stressors, they can change their 
response to stressors. 



Lifestyle Changes 

5. Intentionally changing the stressful aspects of one's Jifestyle or environment can help 
one cope with psychosocial stressors such as frustration, crowding, overload, 
deprivation, noise, and life changes. 

6. Improving one's skills can reduce the stress caused by lack of ability. 

7. An effective way to reduce stress is to find alternatives for goals and behaviors that 
one has been unable to accomplish. 

8. Avoiding crowded situations whenever possible is an effective technique for reducing 
the stress of crowding. 

9. Learning to say "no" to requests for time when one already has enough to do reduces 
stress. 

10. Delegating authority and responsibility to others can help reduce stress. 

11. Stress can be reduced by breaking down a long and complicated task into smaller 
parts. 

12. Time management reduces stress by matching things that need to be accomplished 
with the time available. 

13. Listing tasks in order of their importance so that the most important tasks can be 
completed first helps to reduce stress. 

14. Accepting the fact that no one can do everything perfectly helps to reduce stress. 

15. Stress reduced by anticipating periods of boredom and planning something 
stimulatmg to do during those periods. 

16. Wearing earplugs is an effective way to cut down on the stress caused by loud arid/or 
annoying noise. 

17. When undergoing important life changes, stress can be minimized by reducing me 
number of other changes that are made. 
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18. Vacations that involve dramatic changes in location, routine, or level of stimulation 
often increase stress* 

19. Stress can be reduced by establishing routines that become automatic. 

20. Receiving accurate information and having the ability to do something about a 
stressful situation can reduce one's stress level. 

Personality Changes 

2L Intentionally changing the stressful aspects of one's personality can help one cope 
with symbolic stressors such as unwarranted fears or unpleasant thoughts. 

22. Improving one's self-concept reduces stress. 

23. Becoming more assertive and focusing on the positive aspects of oneself and one's 
life can improve one's self-concept. 

24. Thought-stopping is a technique whereby a stressed individual intentionally stops 
thinking negative, persistent thoughts. 

25. Developing close friendships with people one can trust reduces stress. 

26. Type A behavior can be reduced by becoming less competitive with oneself and 
others, concentrating on one project at a time, and slowing down. 

Relaxation 

27. Physiologically, relaxation is the opposite of the stress response. 

28. A decreased heart rate, decreased blood pressure, decreased rate of respiration, and 
decreased muscle activity are characteristics of relaxation. 

29. By producing one characteristic of relaxation, an individual can start the chain of 
physiological changes that occur when one relaxes. 

30. Relaxation replaces the usual thought processes with an altered state of 
consciousness. 

31. Being comfortable, breathing deeply, and assuming a quiet, accepting attitude 
encourages relaxation. 

32. Relaxation can be used in short spurts and almost anywhere. 
Relaxation Training 

33. Relaxation training allows people to regulate bodily processes that were formerly 
considered beyond conscious control. 

34. Relaxation training helps people become more aware of their bodies and how they 
respond to stress. 

35. Biofeedback, autogenic training, progressive relaxation, and meditation are 
relaxation training techniques. 
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36. Biofeedback is the use of an instrument which alerts the individual to subtle changes 
in bodily processes. 

37. Muscle contraction, brain activity, heartbeat, and blood pressure are bodily processes 
that can be monitored by biofeedback. 

38. Once people have learned hov/ to control bodily processes using biofeedback 
instruments, they usually can duplicate the results when not monitored by the 
instruments. 

39. Biofeedback is an effective technique for relieving muscular problems and other 
disorders associated with stress. 

40. Autogenic training is a form of relaxation training which uses self-directed images of 
relaxed states. 

41. In autogenic training, one attempts to produce physical sensations that are associated 
with relaxation. 

42. Imagining that one's body parts are warm and heavy is a technique used in autogenic 
training. 

43. Autoge*iic training is an effective technique for relieving high blood pressure and 
other vascular problems associated vnth stress. 

44. In progressive relaxation, one learns to relax muscles by first learning what it feels 
like to be tense and then what relaxation feels like. 

45. Progressive relaxation is an effective technique for relieving headaches, backaches, 
and other muscular problems caused by stress. 

46. In one form of meditation, attention is focused on one object, such as a word, sound, 
or one's own breath. 

47. A quiet internal and external environment are essential for effective meditation. 

48. Sitting in a comfortable position and wearing loose clothing help to quiet one's 
internal environment. 

49. Being in a location where one won t be disturbed helps to quiet one's external 
environment. 

Physical Activity 

50. Physical activity that is not competitive is an effective stress management technique. 

51. When using physical activity as a stress management technique, one should be aware 
of how the activity is affecting one's body and be careful not to o^'erexert. 

52. Physical activity can be used to reduce stress in the body, to decrease one's reactivity 
to future stress, and to promote feelings of well-being and inner calm. 

53. Swimming, running, dancing, biking, and other individual sports are appropriate 
physical activities for reducing stress in the body. 

54. A person who uses physical activity to reduce stress in the body should i ecognize 
when stress occurs and pursue physical activity soon afterwards. 
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55. A regular physical activity program can decrease the effects of future stress. 

56. Physical activity that is vigorous enough to bring relaxation afterwards makes a 
person less open to the negative effects of stress. 

57. Becoming involved in the joy of physical activity leads to feelings of well-being. 

Diet Management 

58. One should consume less caffeine, especially during stressful times. 

59. Getting enough vitamins, especially the B vitamins and Vitamin C, is important 
during stressful times. 

60. B vitamins are found in breads, beans, nuts, and meats; Vitamin C is found in fruits 
and vegetables. 
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Appendix B 



INFORMED CONSENT PROCEDURES 

Prior to administering measures to participants, program personnel should inform 
participants about the content covered by the measures and the purpose of the program's 
evaluation study. Program personnel may also wish to provide the opportunity for 
participants to indicate whether or not they consent lu ^participate in the study and complete 
the selected measures. Informed consent is obtained by presenting all information pertinent 
to the study and asking the participant to affix a signature indicating that the information has 
been read and that consent is given to participate. 

If the decision is made to obtain informed consent, program personnel have the choice of 
employing a "passive'' consent procedure or an "active" consent procedure. Passive 
informed consent consists of asking participants to sign and return a consent form only if they 
do not wish to participate in the study. Participants who do not return the consent form aie 
considered eligible to participate in the study. 

Active informed consent requires participants to sign and return the consent form if they 
wish to participate. Only those participants who return a signed fonn can be included in the 
study. Consequently, the participation rate resulting from an active consent procedure is 
generally lower than that obtained from a passive consent procedure. 

To construct an informed consent form, program personnel should consider including ihe 
following items: 

1. A general statement of the program goals and objectives. 

2. A brief explanation of the study procedures and measures. 

3. An indication that the participant is free to withdraw consent and to 
discontinue participation at any time. 

4. An explanation of the procedures to be taken to ensure aionymity and 
confidentiality of responses. 

5. An indication that participants arc free not to answer specific items or 
questions. 

6. A place for the participants to affix their signatures under a statement 
indicating that the participant agrees to participate (active consent) or does 
not agree to participate (passive consent) in the study. If appropriate, a date 
for the return of the consent form should be specified. 
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Appendix C 



ANNOTATED EVALUATION BIBLIOGRAPHY 

1. Alkin, M.C., & Solmon, L.C. (Eds.). (1983). The costs of evaluation. Beverly Hills, CA: 

Sage. 

In this collection of essays both theoretical and practical issues rsltvant to cost-focused program 
evaluations are presented. 

2. American Psychological Association. (1973). Ethical principles in the conduct of 

research with human participants. Washington, DC: Author. 

This treatise focuses on the appropriateness of carrying out various types of research 
investigations with human su^'scts. Because the American Psycholo^cal Association has had a 
long-standing concern abow, ethical issues in the conduct of research investigations, this 
publication will be of interest to numerous evaluators of health education programs. 

3. American Psychological Association, American Educational Research Association, 

National Council on Measurement in Education. (1985). Standards for educational 
and psychological tests. Washington, DC: Author. 

This volume presents the most widely used set of standards for psycholo^cal and educational 
tests. Frequently cited by users of educational tests, the standards have recently been employed 
in numerous judicial deUbt rations. Relatively brief, the standards should be consulted by health 
educators who employ assessment devices regularly. 

4. Anderson, L.W. (1981).. issessing affective characteristics in the schools. Boston: Allyn 

and Bacon. 

Anderson provides an excellent set of practical suggestions for the creation of affective 
assessment instruments. He includes one of the most easily understood expositions of various 
scaling procedures including Likert, Thurstone, and Guttman scales. 

5. Bausell, R.B. (Ed.). Evaluation and the health professions. Newbury Park, CA: Sage. 

This quarterly publication deals with a variety of evaluation relevant issues of interest to health 
educators. 

6. Berk, R.A. (Ed.). {l%A).A^ide to criterion-referenced test construction. Baltimore: 

The Johns Hopkins University Press. 

This collection of essays consists of papers presented at the first Johns Hopkins University 
National Symposium cn Educational Research. In addition, a number of more recently written 
chapters have been included in this revision of a 1980 text.The authors treat many of the important 
problems, both conxptual and technical, facing developers and users of criterion-referenced 
measures. 
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7. BerK, R.A. (Ed.). (1982). Handbook of methods for detecting test bias. Baltimore: The 
Johns Hopkins University Press. 



This collection of Individual essays offers the reader a comprehensive depiction of methods 
currently available to detect the presence of bias in tests. 



8. Campbell, DX, & Stanley, J.C (1966). Experimental and quasi-experimental designs 
for research. Chicago: Rand McNally* 

This volume, originally a chapter in a larger volume, has had substantial impact on the fields of 
research and evaluation. Evaluators of health education programs will wish to consider tlus truly 
classic treatment of data-gathering designs suitable for experimental and quasi-experimental 
settings. 



9. Churchill, G.A., Jr. (1979). Marketing research: Methodolo^cal foundations (2nd ed.). 
Hinsdale, IL: The Dryden Press. 



Although written in the context of marketing research, this textbook covers several topics of vital 
importance in evaluation. Topics such as research design, data collection, sampling, and data 
analysis are covered in a readily understandable yet accurate way. An excellent resource. 



10. Cohen, L (1977). Statistical power analysis for the behavioral sciences (Rev. ed.). New 

York: Academic Press. 

Cohen offers a useful treatment of factors which should be considered when one draws samples 
for use in research or evaluation activities. Of special interest is the set of easy-to-use guidelines 
he offers for determining the estimated sample size necessary to detect differences between 
groups. 

11. Cook, T.D., & Campbell, D.T. (1976). The design and conduct of quasi-experiments 

and true experiments in field settings. In M.D. Dunnette (Ed,), Handbook of 
industrial and organizational psychology. Chicago: Rand McNally. 



This is an updated version of the famous exposition of quasi-experimental and experimental 
data-gathering designs by Donald T. Campbell and Julian C. Stanley (see Reference No. 8). An 
excellent discussion of four types of validity is featured in this essay. 



12. Cook, T.D., & Campbell, DX (1979). Quasi-experimentation: design & analysis issues 
for field settings. Chicago: Rand McNally. 



This widely cited volume provides a comprehensive treatment of quasi-experimental 
investigations in settings of substantial relevance to the concerns of health educators. There are 
excellent discussions of Internal and external vab'dity, including the various threats to both types 
of validity. A systematic consideration of the commonly used data-gathering designs is offered, 
including an extended appraisal of interrupted time-series designs. 
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13. Cordray, D.S., Bloom, H.S. & Light, RJ. (Eds.). (1987, Summer). Evaluation practice 

in review (New Directions for Program Evaluation, No. 34). San Francisco: 
Jossey-Bass. 

This volume contains a set of thought-provoking chapters dealing with what has been learned 
about the practice of evaluation during the past decade. The chapters on evaluation politics by 
Eleanor Chelimsky and on naturalistic evaluation by Egon Guba would be a particular interest 
to evaluators of health education programs. 

14. Cronbach, L.J, (1963). Course improvement through evaluation. Teachers College 

Record, 64, 672-683. 

This article is an earl^ piece, presentmg the virtues of what would later be termed "formative" 
evaluation. It rings as true today as it did more than two decades ago, and it applies as much to 
evaluation in health education as it does to more traditional evaluation. Emphasizing the role of 
evaluation in gathering information that can unprove programs, this article is well worth reading. 

15. Cronbach, L.J., Ambron, S.R., Dornbusch, S.M., Hess, R.D., Hornik, R.C., Phillips, 

D.C., Walker, D.F., & Weiner, S.S. (1980), Toward reform of program evaluation. 
San Francisco: Jossey-Bass. 

This important book considers the function of evaluation in a pluralistic society and presents 95 
theses on the role of evaluators and evaluations. In addition to providing a contemporary 
conception of evaluation, it provides a historical and multidisciplinary perspective of the field. 
This volume will be of considerable interest to those evaluating health education programs. 

16. Cronbach, L.L (1977). Analysis of covariance in nonrandomized experiments: 

Parameters affecting bias. Unpublished occasional paper, Stanford Evaluation 
Consortium, Stanford University, 

A highly technical piece on the complications associated with using analysis of covariance, this 
article is recommended only for those prepared to handle a critical data- analysis problem in a 
sophisticated way. 

17. Cronbach, LX, & Furby, L. (1970). How should we measure 'change' — or should we? 

Psychological Bulletin, 74, 68-80. 

A technical treatise on the dangers associated with using gain scores. A very significant piece, but 
recommended only for those with some psychometric training. 

18. Cumiingham, G.K. (1986). Educational and psychological measurement New York: 

Macmillan. 

This is a standard introductory text focusing on the major topics associated with measurement as 
it applies to such tasks as program evaluation. 

19. Ebel, R.L. (1979). Essentials of educational measurement (3rd ed.). Englewood Cliffs, 

NJ: Prentice-Hall. 

This is a standard, easily read introductory texl, covering important topics in the field of 
educational testing. Ebel, a prominent leader of traditional educational testing practices, provides 
a lucid treatment of a wide range of measurement topics. 
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20. Fetterman, D.M. & Pitman, M.A. (Eds.). (1986).Educational evaluation: Ethnography 

in theory, practice, and politics. Beverly Hills, CA: Sage. 

This collection of essays touches on ethnographically oriented evaluation of educational 
programs. Health educators wishing to learn about this recently emphasized approach to 
educational evaluation will find this volume of interest. 

21. Green, L.W. (1979). Research methods translatable to the practice setting: From rigor 

to reality and back. In S.J. Cohen (Ed.), New directions in patient compliance 
(pp. 141-151). Lexington, MA: Lexington Books. 

Green attends to a practical dilemma facing those who evaluate health education programs, 
namely, the necessity to make trade-offs between validity and feasibility in field settings. Six 
strategies for coping with evaluation under adverse circumstances are described. 

22. Green, L.W., & Figa-Talamanca, L (1974). Suggested designs for evaluation of patient 

education programs. Health Education Myographs, 2 (1), 54-71. 

In this essay Green and Figa-Talamanca suggest data-gathering designs for conducting 
evaluations of patient edua tion programs. The authors also explore several issues related to 
evaluations of this variety. 

23. Green, L.W., & Lewis, F.M. (1986). Measurement and evaluation in health education 

and health promotion. Palo Alto, CA: Mayfield. 

This volume is an excellent resource for health educators concerned with the evaluation of their 
programs. Green and Lewis provide a series of useful explanations of topics in both measurement 
and health evaluation. Their expositions are peppered with practical examples drawn from health 
education and health promotion. 

24. Hambleton, R.K., Swaminathan, H., Algina, J., & Coulson, D.B. (1978). 

Criterion-referenced testing and measurement: A review of technical issues and 
development. i?ev/ew of Educational Research, 48 (1), 1-48. 

This is a comprehensive review of the field of criterion-referenced testing. Hambleton and his 
colleagues do a masterful job of isolating the key issues m criterion-referenced testing and 
describing results of res earch investigations bearing on those issues. Somewhat technical at times, 
this review is one of the more widely cited essays dealing with criterion-reierenced testing. 

25. Hays, W.L. (1973). Statistics for the social sciences. New York: Holt, Rinehart, and 

Winston. 

This comprehensive text handles basic and advanced statistical considerations. Somewhat 
technical at points. Hays nonetheless provides an excellent set of step-by-step guidelines to 
statistical practice. 
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26. Joint Committee on Standards for Educational Evaluation. (1981). Standards for 

e\'aluations of educational programs, projects, and materials. New York: 
McGraw-Hill. 

The development of these evaluation standards was spearheaded by a joint committee of the 
American Educational Research Association, the American Psychological Association, and 
the National Council on Measurement in Education. Thirty standards are presented, 
addressing issues related to deciding whether to evaluate, defining the evaluation problem, 
designing the evaluation, budgeting for the evaluation, collecting and analyang data, and 
reporting the evaluation. Intended for both consumers of evaluation and individuals 
conducting evaluations, this reference may be of most use to evaluators who arc relatively new 
to the field. 

27. Kubiszyn, T., & Borich, G. (1987). Educational testing and measurement: Classroom 

application and practice (2nd ed.). Glenview, IL: Scott-Foresman. 

Another introductory text dealing with the nuts and bolts of measurement, this book vwll 
provide health educators with a good overview of educational measurement. 

28. Levin, H.M. (1975). Cost-effectiveness analysis in evaluation research. In M. 

Guttentag & E.L Struening (Eds.), Handbook of evaluation research (Vol.2, 
pp.89-122). Beverly Hills, CA: Sage. 

This essay probes the important considerations involved in determining cost-effectiveness of 
programs in the context of educational evaluations. Theoretical as well as practical guidelines 
are provided. 

29. Levin, H.M. (1983). Cost-effectiveness: A primer (New Perspectives in Evaluation, 

Vol.4). Beverly Hills, CA: Sage. 

This text is a splendid introduction to the fundamental concepts of cost analysis on program 
evaluation. Levin provides succinct descriptions along wth advantages and disadvantages for 
cost-feasibility, cost-effectiveness, cost-benefit, and cost-utility analyses. 

30. Linn, R.L., & Slinde, J.A, (1977), The determination of the significance of change 

between pre- and post- testing periods. Review of Educational Research, 47, 
12M50. 

This article reviews many of the major issues in the measurement of change from pretesting to 
posttesting periods and suggests possible alternatives. These authors share the general 
sentiment of many others in the field that, "more is expected from gain scores than they can 
reasonably be expected to provide." 

31. Lx)rd, F.H. (1963), Elementary models for measuring change. In C.W. Harris (Ed.), 

Problems in measuring change (pp.21-38). Madison: Wisconsin Press. 

This is an early treatise on the problems associated with measuring change. Although this 
chapter rapidly becomes very technical, the early sections provide an intuitive explanation of 
the difficulties with using gain scores. 
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32. Mark, M.M., & Shetland, RX. (Eds.). (1987, Fall). Multiple methods in program 

evaluation (New Directions for Program Evaluation, No. 35). San Francisco: 
Jossey-Bass. 

Decrying the infrequency with which multiple methods are used in program evaluation, six 
chapters are offered in this volume, not only advocating multiple methods, but also describing 
how such program evaluations can be conducted. 

33. Oakland, T. (Ed.). (1977). Psychological and educational assessment of minority 

children. New York: Brunner/Mazel. 

This collection of essays provides a series of useful suggestions for those who are more sensitive 
to the possible bias present In educaHonal tests. 

34. Popham, W.J. {19SS). Educational evaluation. Englewood Cliffs, NJ: Prentice-Hall. 

This is an introductory text, written in tcJrly non technical language, about the field of educational 
evaluation. Evaluators of Lalth education programs will find it simple to translate the book's 
contents to their own spedalties. 

35. Popham, W.J. (1981). Modem educational measurement. Englewood Cliffs, NJ: 

Prentice-Hall. 

Varied topics in the field of educational measurement are introduced in this text. 
Norm-referenced measurement and criterion-referenced measurement are both considered, 
with the special applications of criterion-referenced assessment emphasized. Chapters on the 
relationship of testing to teaching and the measurement of affect will be of special Interest to 
health educators. 

36. Popham, W.J., & Sirotnik, K.A. (1973). Educational statistics: Use and interpretation 

(2nd ed.). New York: Harper and Row. 

This easily read Introductory text deals with the fundamental types of statistical considerations 
needed by program evaluators. It Is Intended for those who are not particularly comfortable with 
mathematical approaches to statistics. 

37. Riecken, H.W., & Boruch, R.R (1971). Social experimentation: A method for planning 

and evaluating social intervention. New York: Academic Press. 

This Is a significant contribution to our thinking about large-scale social Interventions, their design 
and appraisal, provides a useful analysis of the ways that the experimental method can be 
defenslbly employed In connection with major social programs. 

38. Rivlin, A.M., & Timpane, RM. (Edo.). (1975). Ethical and legal issues in social 

experimentation. Washington, DC: Brookings Institution. 

Rivlin and Timpane explore the sorts of legal and ethical issues to which evaluators of health 
education programs must attend. 

39. SPSS-X User's Guide (3rd ed.). (1988). Chicago: SPSS Inc. 

This is a widely used, well organized set of "canned" computer analysis programs for use In the 
social sciences. Health educators who have occasion to use computer ^.nalyses will find the SPSS 
manual most helpful. 
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40. Salvia, J., & Ysseldyke, J.R (1981)yissessment in special and remedied education (2nd 

ed.) Boston: Houghton Mifflin. 

This text, intended for individuals who must apply assessment to special education and remedial 
education, provides measurement insights for health educators who deal with such populations 
of learners. 

41. Scriven, M. (1967). The methodology of evaluation. In R.W. Tyler, R.M. Gagn6, & 

M. Scriven (Eds.), Perspectives of curriculum evaluation (pp. 39-83), Chicago: Rand 
McNally. 

This seminal article was the first essay in which Scriven distinguished between the now commonly 
accepted formative and summative roles of evaluators. Scriven ranges over a wide variety of 
topics, emphasizing the importance of comparative appraisals of two or more programs' merits. 

42. Scriven, M. (1972). Prose and cons about goal-free evaluation. Evaluation Comment, 

5, 1-4. 

In this essay Scriven offers goal-free evaluation as an antidote to excessive preoccupation with 
the program staffs expressed objectives. Scriven argues that evaluators should attend to the 
results produced by a program, not the rhetoric of its program goals. 

43. Siegel, S. (1956). Nonparametric statistics for the behavioral sciences. New York: 

McGraw-Hill. 

This is the classic treatment of nonparametric statistical techniques. Although a bit out of date 
these days, Siegefs text offers the most easily understood treatment of nonparametric statistical 
procedures. Because of the author's admitted zealousness in support of nonparametric 
techniques, those using Siegel's text should also consult a critique of it by Robert Savage, /owmj/ 
of American Statistical Association, 1957,52, 331-344. 

44. Suchman, E.A. (1967). Evaluative research: Principles and practice in public service 

and social action programs. New York: Russell Sage Foundation. 

In this volume, Suchman provides extensive coverage of the application of the experimental 
research model in conducting evaluations. Although evaluation has come a long way since this 
book was written, the volume provides a clear description of the predominant conceptualization 
of evaluation in the past decade. 

45. Tukey, J.W. (1977). Exploratory data analyses. Reading, MA: Addison-Wesley. 

Creative approaches to displaying and understanding data are provided by Tukey in this excellent 
demystification of data analysis. 

46. Walberg, H.J., Postlethwaite, T.N., Creemers, B.P.M., & de Court, E. (Eds.). (1987). 

Educational evaluation: The state of the field. International Journal of Educational 
Research, 11 (1). 

This special issue, as its title suggests, presents comprehensive revaew of field of program 
evaluation from authors based in the U.S. and abroad. 
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47. Webb, EJ., Campbell, D.T., Schwartz, R.D., Sechrest, L., & Grove, J.B. (1981). 

Nonreactive measures in the socid sciences (2nd ed.). Dallas: Houghton Mifflin. 

This charming volume provides readers with a series of powerful and clever tactics to secure data, 
particularly of an affective nature, without sensitiang respondents to the evaluator's purposes. 

48. Weiss, CH. (1972). Evaluation research: Methods of assessing program effectiveness. 

Englewood Cliffs, NJ: Prentice-Hall. 

Weiss offers a pithy overview of prominent program evaluation considerations including the 
formulation of questions to be addressed, the design of the evaluation study, and the utilization 
of evaluation results. A papeiback, this brief book (160 pp.) offers an excellent introduction to 
what Weiss refers to as "evaluation research." 

49. Windsor, R.A., Baranowski, T., Clark, N., & Cutter, G. (1984). Evaluation of health 

promotion and education programs. Palo Alto, CA: Mayfield. 

This text is a useful introduction to the evaluation of health education programs. Windsor et al. 
have provided readers with a series of Lealth-relevant examples to illustrate their explorations. 

50. Worthen, B.R., & Sanders, J.R. (1973). Educational evaluation: Theory and practice. 

Worthington, OH: CA. Jones. 

This volume was one of the earliest compilations of various program evaluation models applied 
to education. Evaluation theorists whose views are presented in this book include Stake, 
Cronbach, Scriven, Tyler and others. Worthen and Sanders have authored sections of the book 
and have included a series of ori^nal chapters by a number of evaluation specialists. While 
focused on educational evaluation in general, the volume is of substantial relevance to program 
evaluation of health education programs. 

51. Worthen, B.R., & Sanders, J.R. (1987). Educational evaluation: Alternative 

approaches and practical guidelines. New York: Lx)ngman. 

This introductory text is organized around a scries of alternative approaches to educational 
evaluation, including the "objectives-oriented" and "ad\isory-oriented" approaches, 

52. Worthen, B.R., & White, K.R. (1987). Evaluating educational and social j^rograms: 

Guidelines for proposal review, onsite evaluation, evaluation contracts, and technical 
assistance. Boston: Kluwer-Nijhoff. 

This volu -provides a first-rate series of practical guidelines dealing with varied aspects of 
proposal iv onsite evaluation, evaluation contracts, and technical assistance. 

53. Zdep, S.M., & Rhodes, I.N. (1977). Making the randomized response technique work. 

The Public Opinion Quarterly, 40, 531-537. 

This easily read essay describes the randomized response technique, a procedure used to obtain 
sensitive information from respondents more accurately than if respondents vtcrc directly asked 
about sensitive information. 
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